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EXECUTIVE SUMMARY 


Executive Summary 

In this study, we examined the academic performance of 18 urban districts that participated in 
the 2009 Trial Urban District Assessment (TUDA) of the National Assessment of Education 
Progress (NAEP). The districts participated in grade 4 and grade 8 reading and mathematics 
assessments. Eleven of these districts also participated in the 2007 TUDA. We examined the 
changes in student performance in these 11 districts from 2007 to 2009. 

Our analyses focused on the following questions: 

• How did each district perform in 2009— 

o compared to the national public sample and the large city populations? 
o compared to one another when we control for relevant student background 
characteristics? 

o compared to their expected performance based on relevant student background 
characteristics? 

o across mathematics and reading subscales? 
o at the item level? 

• How did each district’s performance change from 2007 to 2009? 

In the District Profiles section of this report, we answer these questions and also provide relevant 
fiscal and non-fiscal information on each district. 

District Performance Compared to National Public (NP) and Large Cities (LC), 
2009 

In order to describe the most recent performance of the 18 districts on NAEP grade 4 and 8 
reading and mathematics, we computed their average scores in 2009 and compared the average 
score of each district to the national public school sample and the large city (LC) averages. 

In the reading assessment, Charlotte performed above the national public and the LC averages at 
grade 4. 

Average scores for students in Austin, Jefferson County, Miami-Dade County, and New York 
City were not significantly different from the national average at grade 4. While none of the 
districts performed above the national public average at grade 8, scores for students in Austin 
and Miami-Dade County were not statistically different from the national public averages at 
grade 8. 

Furthermore, when compared to the LC grade 4 and 8 reading averages, scores were higher in 
Austin, Boston, Charlotte, Jefferson County, and Miami-Dade County. Scores for New York 
City were higher than the LC average at grade 4 and no different from it at grade 8. In addition, 
average scores in Atlanta, Houston, and San Diego were not significantly different from the LC 
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average at grade 4; and average scores in Atlanta, Chicago, Houston, New York City, 
Philadelphia, and San Diego were not significantly different than the LC average at grade 8. 

In the mathematics assessment, only Charlotte performed above the national public and the LC 
averages at grade 4 and only Austin performed above the national public and the LC averages at 
grade 8. 

When compared to the national average in mathematics, average scores in Austin, New York 
City, and San Diego were no different at grade 4. The same was true for Boston, Charlotte, and 
San Diego at grade 8. 

Furthermore, students in Austin, Boston, Charlotte, Houston, and San Diego outperformed their 
LC peers in mathematics in both grades 4 and 8. On the other hand, average scores for students 
in Miami-Dade County and New York City were higher than the LC average at grade 4 but no 
different from it at grade 8. Finally, Jefferson County students’ average scores were not 
significantly different from the LC average at both grade 4 and grade 8. 


FIGURE 1. Grade 4 Average Reading Scale Figure 2. Grade 8 Average Reading Scale 
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Figure 3. Grade 4 Average Mathematics 
Scale Scores for TUDA Districts, LC and 
NP, 2009 


Figure 4. Grade 8 Average Mathematics 
Scale Scores for TUDA Districts, LC and 
NP, 2009 
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District Performance Compared to Other Districts After Adjusting for Student 
Background Characteristics, 2009 

In addition, we compared the performance of each district against the other districts after 
adjusting (or controlling) for a number of relevant student background characteristics. We 
estimated the performance of each district had its demographic profile been the same as the 
average profile of all 18 districts on relevant student background variables. These analyses put 
the districts on a more level playing field. With these controls, the highest performers were 
Austin, Boston, Charlotte, Miami-Dade County, and New York City in grade 4 reading; Austin, 
Boston, and Miami-Dade County, in grade 8 reading; Austin, Boston, Charlotte, Houston, and 
New York City in grade 4 mathematics; and Austin and Boston in grade 8 mathematics. 
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Table 1. Top-Performing Districts After Adjusting for Student Characteristics, 2009 


Grade 4 

Grade 8 

Reading 

Austin, Boston, Charlotte, 
Miami-Dade County, New York 
City 

Austin, Boston, Miami-Dade 
County 

Mathematics 

Austin, Boston, Charlotte, New 
York City, Houston 

Austin, Boston 


District Expected Performance Compared to Actual Performance, 2009 

We also computed the expected performance of each district based on its profile in terms of the 
selected student background characteristics. Next, we compared each district’s actual 
performance to the expected performance for that district. In grade 4 reading, six districts 
performed higher than expected: Austin, Boston, Charlotte, Houston, Miami-Dade County, and 
New York City. In grade 8 reading, five districts performed higher than expected: Austin, 
Boston, Charlotte, Houston, and Miami-Dade County. In both grade 4 and grade 8 mathematics, 
six districts performed higher than expected statistically: Austin, Boston, Charlotte, Houston, 
Miami-Dade County, and New York City. 

Table 2. Districts Performing Higher Than Expected Based on Selected Student Backgrounds, 
2009 



Grade 4 

Grade 8 


Austin, Boston, Charlotte, 

Austin, Boston, Charlotte, 

Reading 

Houston, Miami-Dade County, 
New York City 

Houston, Miami-Dade County 


Austin, Boston, Charlotte, 

Austin, Boston, Charlotte, 

Mathematics 

Houston, Miami-Dade County, 

Houston, Miami-Dade County, 


New York City 

New York City 


District Performance Across Subscales, 2009 

In addition to comparing each district’s average scale scores to other districts and to the national 
public and LC averages, we looked at the relative performance of each district across subscales. 
The 2009 reading assessment had two subscales: reading for a literary experience and reading for 
information. The mathematics assessment for the same year had the following subscales: number 
properties and operations, measurement, geometry, data analysis and probability, and algebra. 
Note that the NAEP subscales are not reported on the same metric; hence the subscale means are 
not directly comparable. Instead, we conducted normative comparisons between subscales 
(within a district) by looking at the percentile that a given district’s subscale mean corresponded 
to on the score distribution of the national public school sample. 

In reading, the differences between the percentiles for the two subscales were relatively small. At 
grade 4, only Boston and Fresno had differences of five or more percentage points. At grade 8, 







Jefferson County had the largest difference at five percentage points. Across districts and 
subscales, Detroit showed the weakest performance in reading, with average performance on the 
information subscale corresponding to the 16 th percentile (grade 4) and 17 th percentile (grade 8) 
on the score distribution of the national public school sample. 

The range among the percentiles for the five subscales in mathematics was wider than the range 
in reading. At grade 4, seven districts showed differences of 10 or more percentage points: 
Baltimore City, Boston, Charlotte, Fresno, Jefferson County, Miami-Dade County, and San 
Diego. At grade 8, only, Austin and Charlotte had a range of 10 or more percentage points. 
Across districts and subscales, Detroit showed the weakest performance in mathematics with its 
average performance on the number subscale in grade 4 corresponding to the 9 th percentile on the 
score distribution of the national public school sample. At grade 8, this district’s average 
performance on the measurement, geometry, and data subscales was at the 12 th percentile. 

Table 3. Districts with Large Performance Differences Across Subscales, 2009 



Grade 4 

Grade 8 

Reading* 

Boston, Fresno 

Jefferson County 

Mathematics** (Subscale 

Baltimore City, Boston, 

Austin, Charlotte 

difference of at least 10 

Charlotte, Fresno, Jefferson 


percentile points) 

County, Miami-Dade County, 
San Diego 



* Difference of at least 5 percentile points across subscales 
**Difference of at least 10 percentile points across subscales 


District Performance at the Item Level, 2009 

In addition to examining composite and subscale average scores, we looked at district 
performance at the item level. For grade 4 and 8 reading assessments, we computed average 
percent correct (p-values) and average omission-rates by subscale and item type (multiple- 
choice, short constructed-response, and extended constructed-response). For grade 4 and 8 
mathematics assessments we computed average percent correct and average omission-rates by 
subscale, item type, and mathematical complexity (low, moderate, and high). 1 

The average overall percent-correct (p-value) in the grade 4 reading assessment ranged from 38 
percent in Detroit to 59 percent in Charlotte. In fact, Charlotte had the highest and Detroit had 
the lowest average p-values across the two subscales and the three item types (multiple-choice, 
short constructed-response, extended constructed-response). Austin was similar to Charlotte at 
65 percent correct on multiple choice items. Average omission-rates were relatively low across 
all districts except for extended constructed-response (ECR) items in Detroit, where the average 
omission-rate reached 9 percent. 

The picture in grade 8 reading was similar. The average overall p-values ranged from 49 percent 
in Detroit to 65 percent in Austin. Detroit also had the lowest average p-values across all 


1 For more information on mathematical complexity, see Chapter 3 of the Mathematics Framework for the 2009 
National Assessment of Educational Progress at http://nagb.org/publications/frameworks/math-framework09.pdf 
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subscales and item types. Austin had the highest average p-values across subscales and item 
types, with the exception of short constructed-response items, where Jefferson County, Boston, 
and Miami had the highest overall average p-value (55 percent). Average omission-rates for ECR 
items were relatively high, exceeding 10 percent in Baltimore, Boston, Detroit, District of 
Columbia, Houston, Miami-Dade County, New York City, and Philadelphia. Detroit had the 
highest omission-rate for this item type with 15 percent. 

In grade 4 mathematics, the average overall p-values ranged from 32 percent in Detroit to 58 
percent in Charlotte. Charlotte had the highest and Detroit had the lowest average p-values 
across the five subscales, the three item types, and the three mathematical complexity levels 
(low, moderate, and high). The only exception was in measurement, where Austin had the 
highest p-value: 56 percent. Average omission-rates were relatively low across all districts 
except for ECR items, where the average omission-rate reached 8 percent in Los Angeles. 

In grade 8 mathematics, the average overall p-values ranged from 30 percent in Detroit to 54 
percent in Austin. More specifically, Austin had the highest and Detroit had the lowest average 
p-values across the five subscales, the three item types, and the three mathematical complexity 
levels with the exception of short constructed-response where Charlotte had the highest p-value 
at 46 percent. Average omission-rates were relatively high for several districts for ECR items 
and high mathematical complexity items. Atlanta, Baltimore, Boston, Cleveland, Detroit, District 
of Columbia, Houston, Los Angeles, New York City, and Philadelphia all had omission -rates 
exceeding 10 percent for both extended constructed-response items, and high mathematical 
complexity items. The District of Columbia had the highest omission-rate for constructed- 
response items, at 17 percent. 

Table 4. Highest and Lowest Average Percent Correct Rates for Reading and Mathematics, 
2009 


Average Overall Percent Correct 


Highest District 

Lowest District 

Grade 4 Reading 

Charlotte (59%) 

Detroit (38%) 

Grade 8 Reading 

Austin (65%) 

Detroit (49%) 

Grade 4 Mathematics 

Charlotte (58%) 

Detroit (32%) 

Grade 8 Mathematics 

Austin (54%) 

Detroit (30%) 
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Changes in District Performance from 2007 to 2009 

As discussed earlier, we examined the changes in district performance from 2007 to 2009 for the 
1 1 districts that participated in both 2007 and 2009 assessments. 2 We tested whether the changes 
were statistically significant. We also tested whether these changes were significantly different 
from the changes observed in the national public sample and the LC populations for the same 
period. We also computed the effect size corresponding to the change in average scores observed 
from 2007 to 2009. The effect size was computed as the ratio of the change in average scores to 
the standard deviation of the corresponding scale in 2007 for the national public school sample. 

In the composite reading scores at grade 4, Boston, District of Columbia, Houston, and New 
York City posted significant gains from 2007 to 2009. In the effect size measure, Houston 
showed the largest gain, with an effect size of 0.18 in the literary subscale. In other words, the 
change in average score from 2007 to 2009 in Houston was nearly equal to 1/5 of a standard 
deviation on the 2007 national public school score distribution. On the other hand, Cleveland 
showed the largest decrease, with an effect size of -0.20 in the literary subscale. 

In the composite reading scores at grade 8, Atlanta, District of Columbia, and Los Angeles 
posted significant gains from 2007 to 2009. In the effect size measure, Atlanta and Austin 
showed the largest gain, with effect sizes of 0.18, in the information subscale. On the other hand, 
Cleveland showed the largest decrease, with an effect size of -0.23 in the literary subscale. 

In the composite mathematics scores at grade 4, Boston, and District of Columbia posted 
significant gains from 2007 to 2009. In the effect size measure, Boston and the District of 
Columbia showed the largest gains, with an effect size of 0.28 in geometry and algebra, 
respectively. On the other hand, Cleveland showed the largest decrease, with an effect size of 
-0.16 in the data subscale. 

In the composite mathematics scores at grade 8, only Austin, District of Columbia, and San 
Diego showed significant gains from 2007 to 2009. In the effect size measure, San Diego 
showed the largest gain, with an effect size of 0.26 in geometry. On the other hand, Los Angeles 
had the largest decrease, with an effect size of -0.10 in the geometry subscale. 


This report includes charters in the TUDA analysis according to the rules that were in place in 2007 and 2009. 
Beginning in 2009, TUDA samples included only those charter schools that each district included for the purpose of 
reporting Adequate Yearly Progress to the US Department of Education under the Elementary and Secondary 
Education Act. This rule did not exist in 2007, so the TUDA sample that year included charters without this AYP 
distinction. The inclusion or exclusion of charter schools from the TUDA samples did not affect the significance of 
the change in reported scores between 2007 and 2009, with the exception of the District of Columbia. Therefore, we 
removed charters from the District of Columbia sample in both years in order to ensure that the scores in 2007 and 
2009 for DCPS were comparable. 
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Table 5. Districts that Showed the Largest Positive and Largest Negative Changes from 2007 
to 2009 Across NAEP Reading and Mathematics Subscales in Terms of Effect Size 3 



Largest Positive Change Largest Negative Change 



District 

Subscale 

Effect 

size 

District 

Subscale 

Effect 

size 

Grade 4 Reading 

Houston 

Literary 

0.18 

Cleveland 

Literary 

-0.20 

Grade 8 Reading 

Atlanta, 

Austin 

Information 

0.18 

Cleveland 

Literary 

-0.23 

Grade 4 
Mathematics 

Boston, 
District of 
Columbia 

Geometry, 

Algebra 

0.28 

Cleveland 

Data 

-0.16 

Grade 8 
Mathematics 

San Diego 

Geometry 

0.26 

Los 

Angeles 

Geometry 

-0.10 


Final Thoughts 

It is evident that the academic performance of public school students in many of the urban 
districts we examined in this report is nowhere near what we would like it to be. However, the 
story is not uniform across all districts. Some districts, such as Charlotte, Boston, and Austin 
performed at levels similar to, in some cases even higher than, the national average. We also see 
districts that are performing below the large city and national averages, yet are making 
significant progress. The District of Columbia, for example, demonstrated significant gains in 
both grades and subjects. 

On the other hand, some districts have a longer path to travel in order to achieve their targets. For 
example, among the 11 districts that participated in 2007 and 2009 NAEP assessments, 
Cleveland and Chicago were the only two districts that performed lower than the national and the 
large city averages and showed no gains from 2007 to 2009. 

Like several other studies that use NAEP data, this study illustrates the depth and wealth of 
information available about academic performance of public school students in urban districts in 
the United States. Policy makers and practitioners can use this information. The variation in the 
profiles of the 1 8 urban districts examined in this report makes the case that there is much these 
districts can learn from each other. 


3 Largest effect size was determined independent of statistical significance. 
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Pieces of the Puzzle: Recent Performance Trends in Urban Districts — 

A Closer Look at 2009 NAEP Results (An Addendum) 

Background and Purpose 

The purpose of this study was to examine the most recent trends in academic performance in 
reading and mathematics for urban districts participating in the Trial Urban District Assessment 
(TUDA) of the National Assessment of Education Progress (NAEP). Representative samples of 
fourth- and eighth-grade public school students from 18 urban districts participated in the 2009 
reading and mathematics assessments. Eleven of these districts participated in earlier assessment 
years, and seven districts participated for the first time in 2009. Between 800 and 2,400 fourth- 
and eighth-grade students were assessed in each district (NCES, 2010). Table 6 indicates the 
districts that participated in 2007 and 2009 assessments. 

Table 6. 2007 and 2009 NAEP Assessments and TUDA Participation, by District 



2007 

2009 

Districts 

Reading 

Mathematics 

Reading 

Mathematics 

Atlanta 

V 

V 

V 

V 

Austin 

V 

V 

V 

V 

Baltimore City 



V 

V 

Boston 

V 

V 

V 

V 

Charlotte 

V 

V 

V 

V 

Chicago 

V 

V 

V 

V 

Cleveland 

V 

V 

V 

V 

Detroit 



V 

V 

District of Columbia 

V 

V 

V 

V 

Fresno 



V 

V 

Houston 

V 

V 

V 

V 

Jefferson County (KY) 



V 

V 

Los Angeles 

V 

V 

V 

V 

Miami-Dade County 



V 

V 

Milwaukee 



V 

V 

New York City 

V 

V 

V 

V 

Philadelphia 



V 

V 

San Diego 

V 

V 

V 

V 


Research Questions 

We answered the following research questions for each TUDA district for grades 4 and 8, based 
on data from the two most recent NAEP assessments, 2007 and 2009, in reading and 
mathematics: 

1. How did the district perform compared to the national public sample and the large city 
(LC) populations in 2009? 
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2. How did the district perform in 2009, compared to the other districts when we control for 
relevant background variables? 

3. How did the district perform in 2009, compared to their expected performance based on 
relevant background variables? 

4. How did the district’s performance vary across subscales in 2009? 

5. At the item level, what was the average percentage correct for the district in 2009? 

6. In the 2009 assessment, what were the objectives (mathematics) or cognitive targets 
(reading) of the top five differentially most difficult items for the district? 

7. Did the district show significant gain from 2007 to 2009 in terms of overall and subscale 
performance? 

8. What were the changes in the district’s overall and subscale averages from 2007 to 2009 
expressed as effect size? 

We answer these questions in the District Profiles section of this report. 

Methods and Data Analysis 
District Performance in 2009 

In order to describe the most recent performance of the 18 districts on NAEP grade 4 and 8 
reading and mathematics, we first report their average scores and associated standard errors. 
Next, we compare the average score of each district to the national public school sample and the 
large city (LC) averages. We conducted pairwise comparisons to test whether district means 
were significantly different from the national and LC averages. As the number of comparisons 
that are conducted at the same significance level increases, it becomes more likely that at least 
one of the estimated differences will be significant merely by chance. To control for multiple 
comparisons, these analyses were conducted using the Benjamini-Hochberg (1995) false 
discovery rate (FDR) procedure. 

In addition, we compared the performance of each district against the other districts after 
adjusting for certain student background characteristics. These analyses address a particular 
concern raised by many stakeholders when comparisons are made among states or districts with 
differing student background characteristics. A natural question is whether the differences we 
observe would have been different if all the jurisdictions being compared had the same 
demographic profile in terms of relevant student background characteristics. 

Fortunately, we have statistical methods that allow us to make comparisons among states or 
districts by controlling for these characteristics. We conducted regression analyses to estimate 
the performance of a district had its demographic profile, in terms of the selected students 
background characteristics, been the same as the average profile of all 18 districts. These 
analyses put the districts on a more level playing field with regard to these characteristics. 

Based on a literature review, we identified the following NAEP background variables as most 
relevant: race/ethnicity; special education status; English language learner status; indicators of 
the socioeconomic status of students, i.e., eligibility for free- or reduced-price lunch under the 
National School Lunch Program; the highest level of education attained by either parent; and 
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information on the availability of literacy materials and computers in the students’ homes. * 4 We 
identified other studies where similar or identical background variables were used to estimate 
adjusted means. For example, Braun, Je nki ns and Grigg (2006a) examined the differences in 
mean NAEP reading and mathematics scores between public and private schools, adjusting for 
gender, race/ethnicity, disability status, and identification as an English language learner (ELL). 
Braun, Jenkins, and Grigg (2006b) compared charter schools to public schools using the same 
approach. 

Based on the same regression analyses discussed above, we also computed the expected 
performance of each district based on their profile in terms of the selected student background 
characteristics. Next, we compared each district's actual performance to the expected 
performance for that district. We call the difference between the two the "district effect." 5 
Positive effects indicate that the district is performing higher than expected statistically and 
negative effects indicate that the district is performing lower than expected statistically. Note that 
there are limitations to these analyses. The adjusted performance and expected performance are 
both estimated based on variables that may affect student achievement and are beyond the 
control of the educators and policy-makers. 

It is obvious that we do not, and cannot, control for all such variables. There may be other 
variables that are related to achievement that we are not controlling for. Some of these variables 
are not measured in NAEP, and possibly some are not measurable in the first place. District 
effect is a product of our best attempt to estimate if a given district is performing any different 
from expected levels given their student profile on a limited number of variables measured in 
NAEP. 

In addition to comparing each district’s average scale scores to other districts and the national 
public sample and LC averages, we also looked at the relative performance of each district across 
subscales. The 2009 reading assessment included two subscales: reading for a literary experience 
and reading for information. 6 The mathematics assessment for the same year included the 
following subscales: number properties and operations, measurement, geometry, data analysis 
and probability, and algebra. 7 

Note that the NAEP subscales are not all reported on the same metric; hence, the subscale means 
are not directly comparable. Instead, we conducted normative comparisons between subscales 
(within a district) by looking at the percentile to which a given district’s subscale mean 
corresponds to on the score distribution of the national public school sample, for one subscale 
compared to the others. 


See appendix A for information about how the variables we used in the regression analyses were operationally 

defined. 

5 Appendix A illustrates how the ‘district’ effect is estimated based on regression analysis. 

6 We refer to these subscales as literary and information in the remainder of this report. 

7 

We refer to these subscales as numbers, measurement, geometry, data, and algebra in the remainder ol this report. 
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Districts' Item-Level Performance in 2009 

In addition to examining composite and subscale average scores, we looked at each district’s 
performance at the item level. For grade 4 and 8 reading assessments, we computed average 
percent correct (p-values) and average omission-rates by subscale and item type (multiple- 
choice, short constructed-response, and extended constructed-response). For grade 4 and 8 
mathematics assessments we computed average percent correct and average omission-rates by 

Q 

subscale, item type, and mathematical complexity (low, moderate, and high). 

Next, in order to identify items that are differentially more difficult for each district, we 
computed the standardized p-values (in z-score format) for the national public sample and the 18 
districts. An item with a standardized p-value of 0 is an item of average difficulty for the given 
sample. Items with standardized p-values greater than zero are relatively easier for that sample 
and, conversely, items with standardized p-values less than zero are relatively more difficult for 
that sample. Next, we computed the difference between the standardized p-values for the 
national public sample and each district. Larger differences indicate that the item was 
differentially more difficult for the district compared to the nation. We identified the items with 
the largest differences, and we reported the cognitive targets measured by the top five 
differentially most difficult items in grade 4 and 8 reading assessments for each district. For 
grade 4 and 8 mathematics assessments, we listed the objectives measured by the top five 
differentially most difficult items. 

Changes in District Performance from 2007 to 2009 

We examine the changes in district performance from 2007 to 2009 for the 11 districts that 
participated in both assessments. * * 9 We looked at the changes both at the composite and subscale 
levels. We reported if the changes were statistically significant. We tested if these changes were 
significantly different from the changes observed in the national public samples and the LC 
populations for the same period. 

We also computed the effect size corresponding to the change in average scores observed from 
2007 to 2009. The effect size is computed as the ratio of the change in average scores to the 
standard deviation of the corresponding scale in 2007 for the national public school sample. 
Effect size is another measure that allows comparisons across different subscales. 


For more information on mathematical complexity, see chapter 3 of the Mathematics Framework for the 2009 

National Assessment of Educational Progress at http://nagb.org/publications/frameworks/math-framework09.pdf 

9 This report includes charters in the TUDA analysis according to the rules that were in place in 2007 and 2009. 
Beginning in 2009, TUDA samples included only those charter schools that each district included for the purpose of 
reporting Adequate Yearly Progress to the US Department of Education under the Elementary and Secondary 
Education Act. This rule did not exist in 2007, so the TUDA sample that year included charters without this AYP 
distinction. The inclusion or exclusion of charter schools from the TUDA samples did not affect the significance of 
the change in reported scores between 2007 and 2009, with the exception of the District of Columbia. Therefore, we 
removed charters from the District of Columbia sample in both years in order to ensure that the scores in 2007 and 
2009 for DCPS were comparable. 




Results 


The results of all of our analyses are reported mainly in the District Profiles section of this report 
where we answer seven research questions listed above for each district. The following sections 
briefly summarize the overall findings. 

District Performance in 2009 

First, we looked at district performance using average scores on the 2009 NAEP assessments. 

In the reading assessment, Charlotte performed above the national public and the LC averages at 
grade 4. 

Average scores in Austin, Jefferson County, Miami-Dade County, and New York City were not 
significantly different from the national average at grade 4. None of the districts performed 
above the national public average at grade 8. 

Furthermore, when compared to the LC grade 4 and 8 reading averages, average scores were 
higher in Austin, Boston, Charlotte, Jefferson County, and Miami-Dade County. New York City 
scored higher than the LC average at grade 4 and no different from it at grade 8. 

Tables B1 and B2 display the district means and associated standard errors, along with the 
national and LC means for grade 4 and 8 reading assessments. The tables also indicate whether 
the district averages are significantly different from the national public and the LC averages. 

In the mathematics assessment, only Charlotte performed above the national public and the LC 
averages at grade 4 and only Austin performed above the national public and LC averages at 
grade 8. 

Students in Austin, New York City, and San Diego scored no different from the national average 
in mathematics at grade 4. Similarly, students in Boston, Charlotte, and San Diego scored no 
different from the national average at grade 8. 

Lurthermore, students in Austin, Boston, Charlotte, Houston, and San Diego scored higher than 
the LC average in mathematics at both grades 4 and 8. The average scores in Miami-Dade 
County and New York City were higher than the LC average at grade 4 and no different from it 
at grade 8. Linally, Jefferson County’s students performed no different from the LC average at 
grade 4 and grade 8. 

Tables B3 and B4 display the district means, and associated standard errors, along with the 
national and LC means for grade 4 and 8 mathematics assessments. The tables also indicate if the 
district averages are significantly different from the national public and LC averages. 

In terms of the districts’ relative performance compared to each other when we controlled for 
relevant background variables, in grade 4 reading, the highest performers were Austin, Boston, 
Charlotte, Miami-Dade County, and New York City. Similarly, no district performed higher than 
Miami-Dade County, Boston, or Austin in grade 8 reading. Tables Cl and C2 display the relative 
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performance of districts after adjusting for student background characteristics in grade 4 and 8 
reading assessments. 

In grade 4 mathematics, no district outperformed, Austin, Boston, Charlotte, Houston, or New 
York City when we controlled for relevant background variables. Similarly, no district 
performed higher than Austin or Boston at grade 8. Tables C3 and C4 display the relative 
performance of districts after adjusting for student background characteristics in grade 4 and 8 
mathematics assessments. 

Table 7 shows how each district's actual performance compared to its expected performance 
based on its profile on selected student background characteristics. In grade 4 reading, six 
districts performed higher than expected statistically, while nine performed lower. Atlanta, 
Jefferson County, and San Diego were the only three districts that performed no differently than 
expected. Table D1 displays the expected mean and district effects for grade 4 reading for all 18 
districts. 

In grade 8 reading, five districts performed higher than expected statistically, while six 
performed lower. Atlanta, Baltimore City, Chicago, Cleveland, New York, Philadelphia, and San 
Diego were the districts that performed no differently than expected. Table D2 displays the 
expected mean and district effects for grade 8 reading for all 18 districts. 

In grade 4 mathematics, six districts performed higher than expected, while nine performed 
lower. Atlanta, Baltimore City, and San Diego were the only three districts that performed no 
differently than expected. Table D3 displays the expected mean and district effects for grade 4 
mathematics for all 18 districts. 

In grade 8 mathematics, six districts performed higher than expected statistically, while seven 
performed lower. Atlanta, Baltimore City, Chicago, Philadelphia, and San Diego were the 
districts that performed no differently than expected. Table D4 displays the expected mean and 
district effects for grade 8 mathematics for all 18 districts. 

Across grades and subjects, Atlanta and San Diego performed no differently than expected. 
Austin, Boston, Charlotte, Houston, and Miami-Dade County performed consistently higher than 
expected across the four subject and grade combinations. On the other hand, Detroit, District of 
Columbia, Fresno, Los Angeles, and Milwaukee performed lower than expected at both grades 
and in both subjects. 
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Table 7. District Performance Compared to Expected Performance Based on Student 
Background Characteristics, by Subject and Grade: 2009 


Grade 4 


Grade 8 


Districts 

Reading 

Mathematics 

Reading 

Mathematics 

Atlanta 

= 

= 

= 

- 

Austin 

> 

> 

> 

> 

Baltimore City 

< 

= 

= 

= 

Boston 

> 

> 

> 

> 

Charlotte 

> 

> 

> 

> 

Chicago 

< 

< 

= 

= 

Cleveland 

< 

< 

= 

< 

Detroit 

< 

< 

< 

< 

District of Columbia 

< 

< 

< 

< 

Fresno 

< 

< 

< 

< 

Houston 

> 

> 

> 

> 

Jefferson County (KY) 

= 

< 

< 

< 

Los Angeles 

< 

< 

< 

< 

Miami-Dade County 

> 

> 

> 

> 

Milwaukee 

< 

< 

< 

< 

New York City 

> 

> 

- 

> 

Philadelphia 

< 

< 

= 

= 

San Diego 

= 

= 

= 

= 


< District performed lower than statistically expected. 

> District performed higher than statistically expected. 

= District performed no differently than statistically expected. 


Tables El and E2 display the percentiles to which the districts’ overall and subscale performance 
in 2009 correspond to on the national score distribution on the grade 4 and grade 8 reading 
assessments. In reading, the differences between the percentiles for the two subscales were 
relatively small. At grade 4, Boston and Fresno were the two district where this difference was 5 
percentage points or higher. At grade 8, Jefferson County had the largest difference with 5 
percentage points. 

Tables E3 and E4 display the percentiles to which the districts’ overall and subscale performance 
in 2009 correspond to on the national score distribution on the grade 4 and grade 8 mathematics 
assessments. In mathematics, the range among the percentiles for the five subscales was higher 
than in reading. At grade 4, there were seven districts where the range was 10 percentage points 
or higher: Baltimore City, Boston, Charlotte, Fresno, Jefferson County, Miami-Dade County, 
and San Diego. On the other hand, at grade 8, only Austin and Charlotte had a range of 10 
percentage points or higher. 

Districts Item Level Performance in 2009 


In addition to scale scores, we examined each district's performance at the item level. The 
average overall percent-correct (p-value) on the grade 4 reading assessment ranged from 38 
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percent in Detroit to 59 percent in Charlotte. In fact, Charlotte had the highest and Detroit had 
the lowest average p-values across the two subscales and the three item types (multiple-choice, 
short constructed-response, extended constructed-response). Average omission-rates were 
relatively low across all districts except for extended constructed response (ECR) items, where 
the average omission-rate reached 9.4 percent in Detroit at grade 4. Table FI displays the 
average percent-correct and omission-rates for all 18 districts by subscale and item type. 

Grade 8 reading showed a similar picture. The average overall p-values ranged from 49 percent 
in Detroit to 65 percent in Austin. Detroit also had the lowest average p-values across all 
subscales and item types. Austin had the highest overall average p-value (65 percent) and the 
highest average p-values across subscales and item types except SCR items, where Jefferson 
County had the highest overall average p-value (55 percent). For ECR items, average omission- 
rates were relatively high for several districts. Baltimore, Boston, Detroit, District of Columbia, 
Houston, Miami-Dade County, New York City, and Philadelphia all had omission-rates 
exceeding 10 percent for this item type, with Detroit reaching 15 percent. Table F2 displays the 
average percent correct and omission-rates for all 18 districts by subscale and item type. 

In grade 4 mathematics, the average overall p-values ranged from 32 percent in Detroit to 58 
percent in Charlotte. Detroit had the lowest average p-values across the five subscales, the three 
item types, and the three mathematical complexity levels (low, moderate, and high). Charlotte 
had the highest p-values across the board, except in the measurement subscale, where Austin had 
the highest p-value: 56 percent. Average omission-rates were relatively low across all districts 
except for ECR items, where Los Angeles had an 8.4 percent omission-rate in grade 4. Table F3 
displays the average percent-correct and omission-rates for all 18 districts by subscale, item type, 
and mathematical complexity. 

At grade 8 mathematics, the average overall p-values ranged from 30 percent in Detroit to 54 
percent in Austin. Detroit had the lowest average p-values across the five subscales, the three 
item types, and the three mathematical complexity levels. Austin had the highest p-values across 
the board, except in short constructed-response items, where Charlotte had the highest p-value: 
46 percent. Average omission-rates were relatively high for several districts for ECR items and 
high mathematical complexity items. Atlanta, Baltimore, Boston, Cleveland, Detroit, District of 
Columbia, Houston, Los Angeles, New York City, and Philadelphia all had omission-rates 
exceeding 10 percent for both extended constructed-response items and high mathematical 
complexity items. The District of Columbia had the highest omission-rate for extended 
constructed-response items: 17 percent. Table F4 displays the average percent correct and 
omission-rates for all 18 districts by subscale, item type, and mathematical complexity. 

As discussed earlier under the Methods and Data Analysis section, in order to identify items that 
were differentially more difficult for each district, we computed the standardized p-values (in z- 
score format) for the national public sample and the 18 districts. A large positive difference 
between the standardized p-value for the national public sample and a given district for a specific 
item indicates that the item is differentially more difficult for the district compared to the nation. 

In grade 4 reading, Detroit had the item with the largest discrepancy between the standardized p- 
values for the national public sample and the districts. This was a multiple-choice (MC) item in 
the information subscale measuring the cognitive target "integrate and interpret information and 
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ideas presented in text." In the national public sample, 78 percent of the students answered this 
item correctly. In Detroit, 41 percent of students answered the same item correctly. Table G1 
displays the cognitive targets of the top five differentially most difficult items for each district 
measured in grade 4 reading in 2009. 

In grade 8 reading, Atlanta had the item with the largest discrepancy between the standardized p- 
values for the national public sample and the districts. This was again an MC item in the literary 
subscale measuring the cognitive target "integrate and interpret information and ideas presented 
in text." In the national public sample, 67 percent of the students answered this item correctly. In 
Atlanta, 34 percent of students answered the same item correctly. Table G2 displays the 
cognitive targets of the top five differentially most difficult items for each district measured in 
grade 8 reading in 2009. 

In grade 4 mathematics, Cleveland had the item with the largest discrepancy between the 
standardized p- values for the national public sample and the districts. This was a SCR type item 
in the numbers subscale that measured the following objective: Use place value to model and 
describe integers and decimals. In the national public sample, 69 percent of the students 
answered this item correctly. In Cleveland, 32 percent of students answered the same item 
correctly. Table G3 displays the objectives of the top five differentially most difficult items for 
each district measured in grade 4 mathematics in 2009. 

In grade 8 mathematics, Boston had the item with the largest discrepancy between the 
standardized p-values for the national public sample and the districts. This was a MC item in the 
numbers subscale where item measured the following objective: Use place value to model and 
describe integers and decimals. In the national public sample, 66 percent of the students 
answered this item correctly. In Boston, 43 percent of students answered the same item correctly. 
Table G4 displays the objectives of the top five differentially most difficult items for the district 
measured in grade 8 mathematics in 2009. 

Changes in District Performance from 2007 to 2009 10 

In terms of changes in composite reading scores among the TUDA districts at grade 4, Boston, 
District of Columbia, Houston, and New York City posted significant gains from 2007 to 2009. 
In terms of effect size measure, Houston showed the largest gain with an effect size of 0.18 in the 
literary subscale. 11 On the other hand, Cleveland showed the largest decrease, with an effect size 
of -0.20 in the literary subscale. Table HI displays the changes in districts’ overall and subscale 
averages in grade 4 reading assessment from 2007 to 2009 expressed in effect size and whether 
these changes were statistically significant. 


10 This report includes charters in the TUDA analysis according to the rules that were in place in 2007 and 2009. 
Beginning in 2009, TUDA samples included only those charter schools that each district included for the purpose of 
reporting Adequate Yearly Progress to the US Department of Education under the Elementary and Secondary 
Education Act. This rule did not exist in 2007, so the TUDA sample that year included charters without this AYP 
distinction. The inclusion or exclusion of charter schools from the TUDA samples did not affect the significance of 
the change in reported scores between 2007 and 2009, with the exception of the District of Columbia. Therefore, we 
removed charters from the District of Columbia sample in both years in order to ensure that the scores in 2007 and 
2009 for DCPS were comparable. 

11 Largest effect size was determined independent of statistical significance. 




PIECES OF THE PUZZLE 


In grade 8 composite reading scores, Atlanta, District of Columbia, and Los Angeles posted 
significant gains from 2007 to 2009. In terms of effect size measure, Atlanta and Austin showed 
the largest gain, both with an effect size of 0.18 in information subscale. On the other hand, 
Cleveland showed the largest decrease, with an effect size of -0.23 in literary subscale. Table H2 
displays the changes in districts’ overall and subscale averages in grade 8 reading assessment 
from 2007 to 2009, expressed in effect size and whether these changes were statistically 
significant. 

In grade 4 composite mathematics scores, Boston and District of Columbia posted significant 
gains from 2007 to 2009. In terms of the effect-size measure, Boston and the District of 
Columbia showed the largest gain, with an effect size of 0.28 in geometry and algebra 
respectively. On the other hand, Cleveland showed the largest decrease, with an effect size of 
-0.16 in the data subscale. Table H3 displays the changes in the districts’ overall and subscale 
averages in grade 4 mathematics assessment from 2007 to 2009, expressed in effect size and 
whether these changes were statistically significant. 

In grade 8 composite mathematics scores, Austin, District of Columbia, and San Diego were the 
only districts that showed significant gains from 2007 to 2009. In terms of the effect- size 
measure, San Diego showed the largest gain, with an effect size of 0.26 in geometry. On the 
other hand, Los Angeles had the largest decrease, with an effect size of -0.10 in the geometry 
subscale. Table H4 displays the changes in districts’ overall and subscale averages on the grade 8 
mathematics assessment from 2007 to 2009, expressed in effect size and whether these changes 
were statistically significant. Table 3 lists the districts that showed the largest positive and largest 
negative changes across reading and mathematics subscales. 

Table 8. Districts that Showed the Largest Positive and Largest Negative Changes from 2007 
to 2009 Across NAEP Reading and Mathematics Subscales in Terms of Effect Size 



Largest Positive Change 

Largest Negative Change 


District 

Subscale 

Effect 

Size 

District 

Subscale 

Effect 

Size 

Grade 4 Reading 

Houston 

Literary 

0.18 

Cleveland 

Literary 

-0.20 

Grade 8 Reading 

Atlanta, 

Austin 

Information 

0.18 

Cleveland 

Literary 

-0.23 

Grade 4 Mathematics 

Boston, 
District of 
Columbia 

Geometry 

Algebra 

0.28 

Cleveland 

Data 

-0.16 

Grade 8 Mathematics 

San Diego 

Geometry 

0.26 

Los Angeles 

Geometry 

-0.10 
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District Profiles 

In this section of the report, we answer the seven research questions listed earlier for each of the 
18 districts for each grade and subject. The first page of each district profile provides general 
fiscal and non-fiscal information for the district. Non-fiscal information includes the number of 
schools, number of students, student/teacher ratio, and percentage of students in poverty. Fiscal 
information includes total expenditures, instructional expenditures, and expenditures for student 
and staff support, administration, operations, food service and other support staff. We indicate 
the ranking of the district among the 18 examined in this report in terms of student/teacher ratio. 
We also point out the percentage of total expenditures that was instructional. All fiscal and non- 
fiscal information comes from Common Core of Data (CCD) public school district data. 





Atlanta 


Atlanta participated in grade 4 and grade 8 NAEP reading and mathematics assessments in both 
2007 and 2009. It had the third lowest student-teacher ratio among the 18 TUDA districts. Forty- 
nine percent of total expenditures were instructional. 

Table 9. Atlanta's Demographics, 2008-2009 


Number of Schools 

114 

Number of Students 

49,032 

Student/Teacher Ratio 

13.0 

Free and Reduced-Price Lunch 

76% 

Expenditures ($/student) 

Total 

13,516 

Instructional 

6,684 

Student and Staff Support 

1,728 

Administration 

2,252 

Operations, Food Service, and Other Support Staff 

2,853 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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Atlanta: Grade 4 Reading 

• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored lower than five (Boston, Miami-Dade County, New York City, Austin, and 
Charlotte) and higher than nine districts (Baltimore, Chicago, District of Columbia, Los 
Angeles, Milwaukee, Fresno, Philadelphia, Cleveland, and Detroit) in overall reading in 
2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading and in the literary, and information subscales 
corresponded to the 34 th , 36 th , and 33 rd percentiles respectively on the national score 
distribution. The average student was below the national median on all three measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 10. Atlanta's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 



National Public 

Large City 

Atlanta 


Effect Size 

Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

O 

0.04 

O 

0.06 

O 

Information 

0.01 

O 

0.06 

t 

0.04 

O 

Literary 

-0.01 

O 

0.03 

O 

0.07 

O 


Note, f Significant increase, <-> Change not significant, j. Significant decrease 


• On average, students answered 49 percent of the items correctly. The top five 
differentially most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (four items), 
o Locate and recall information from text. 
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Atlanta: Grade 8 Reading 


• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored lower than one (Miami-Dade County) and higher than seven districts (Los 
Angeles, Cleveland, Jefferson County, Milwaukee, District of Columbia, Fresno, and 
Detroit) in overall reading in 2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading and the literary and information subscales in 2009 
corresponded to the 33 ld , 32 nd , and 35 th percentiles, respectively, on the national score 
distribution. The average student was below the national median on all three measures. 

• Displayed significant gain in overall reading and no change in information or literary 
reading subscales from 2007 to 2009. 


Table 11. Atlanta's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Atlanta 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.14 

t 

Information 

0.04 

t 

0.04 

O 

0.18 

O 

Literary 

0.02 

t 

0.05 

t 

0.08 

O 


Note. ] Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 57 percent of the items correctly. The top five 
differentially most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text. 

o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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DISTRICT PROFILES 


Atlanta: Grade 4 Mathematics 

• Scored lower than the national and the LC averages in overall mathematics in 2009. 

• Scored lower than six ( Houston, Austin, Boston, Charlotte, New York City, and Miami- 
Dade County) and higher than nine districts (Philadelphia, Jefferson County, Chicago, 
District of Columbia, Los Angeles, Milwaukee, Cleveland, Fresno, and Detroit) in overall 
mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 30 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
27 th (data) to 33 ld (geometry and algebra). The average student was around the first 
national quartile in data and below the national median on all other subscales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 12. Atlanta's Changes in Grade 4 Mathematics Overall and Subscales Scores, 2007-2009 



National Public 

Large City 


Atlanta 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

0.05 

O 

Numbers 

0.01 

O 

0.05 

t 

0.03 

O 

Measurement 

-0.01 

O 

0.07 

t 

0.14 

O 

Geometry 

0.03 

t 

0.07 

t 

0.05 

o 

Data 

-0.03 

T 

0.01 

O 

-0.14 

o 

Algebra 

0.00 

O 

0.03 

O 

0.10 

o 


Note. ( Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 46 percent of the items correctly. The top five 
differentially most difficult items measured the following objectives: 

o Determine a simple probability from a context, 
o Solve application problems involving numbers and operations, 
o Use informal probabilistic thinking to describe chance events, 
o Order/compare whole numbers, decimals, or fractions. 

o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments 
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Atlanta: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than five (Austin, Boston, Houston, Charlotte, and Miami-Dade County) 
and higher than six districts (Los Angeles, Jefferson County, District of Columbia, 
Milwaukee, Fresno, and Detroit) in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 26 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
23 ld (measurement) to 29 th (algebra and geometry). The average student was below the 
first national quartile in measurement and around the first national quartile in all other 
subscales. 


• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 

Table 13. Atlanta's Changes in Grade 8 Mathematics Overall and Subscale Scores, 2007-2009 



National Public 

Large City 


Atlanta 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

0.09 

O 

Numbers 

0.02 

t 

0.09 

t 

0.04 

O 

Measurement 

0.04 

t 

0.08 

t 

-0.05 

O 

Geometry 

0.05 

t 

0.07 

t 

0.20 

O 

Data 

0.00 

O 

0.02 

O 

0.02 

O 

Algebra 

0.06 

t 

0.05 

o 

0.16 

O 


Note. \ Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 39 percent of the items correctly. The top five 
differentially most difficult items measured the following objectives: 

o Compare objects with respect to length, area, volume, angle measurement, weight, or 
mass. 

o Model or describe rational numbers or numerical relationships using number lines 
and diagrams. 

o Draw or sketch from a written description polygons, circles, or semicircles, 
o Select or use an appropriate type of unit for the attribute being measured such as 
length, area, angle, time, or volume, 
o Construct or solve problems involving scale drawings. 
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Austin 


Austin participated in grade 4 and grade 8 NAEP reading and mathematics assessments in both 
2007 and 2009. It had the sixth lowest student-teacher ratio among the 18 TUDA districts. Fifty- 
seven percent of total expenditures were instructional. 

Table 14. Austin's Demographics, 2009 


Number of Schools 

128 

Number of Students 

83,483 

Student/Teacher Ratio 

14.2 

Free and Reduced-Price Lunch 

63% 

Expenditures ($/student) 

Total 

9,035 

Instructional 

5,156 

Student and Staff Support 

1,105 

Administration 

1,034 

Operations, Food Service, and Other Support Staff 

1,740 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Austin: Grade 4 Reading 

• Scored no differently from the national and higher than the LC averages in overall 
reading in 2009. 

• Scored higher than 12 districts (Atlanta, San Diego, Jefferson County, Baltimore, 
Chicago, District of Columbia, Los Angeles, Milwaukee, Fresno, Philadelphia, 
Cleveland, and Detroit) in overall reading in 2009 after adjusting for relevant background 
characteristics. No district scored higher than Austin in adjusted overall reading scores in 
2009. 

• Average scale scores corresponded to the 46 th percentile in both overall reading and the 
literary subscales, and at the 47 th percentile in the information subscale on the national 
score distribution. The average student was close to the national median on all three 
measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 

Table 15. Austin's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Austin 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.07 

O 

Information 

0.01 

o 

0.06 

t 

0.11 

O 

Literary 

-0.01 

o 

0.03 

O 

0.04 

O 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 57 percent of the items correctly. The top five 
differentially most difficult items measured the cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text (two items). 
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Austin: Grade 8 Reading 


• Scored no different from the national and higher than the LC averages in overall reading 
in 2009. 

• Scored higher than 11 districts (New York City, San Diego, Philadelphia, Baltimore, Los 
Angeles, Cleveland, Jefferson County, Milwaukee, District of Columbia, Fresno, and 
Detroit) in overall reading in 2009 after adjusting for relevant background characteristics. 
No district scored higher than Austin in adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 46 th percentile in both overall reading and the 
information subscale, and the 45 th percentile in the literary subscale on the national score 
distribution. The average student was close to the national median on all three measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 16. Austin's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Austin 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.11 

O 

Information 

0.04 

t 

0.04 

O 

0.18 

O 

Literary 

0.02 

t 

0.05 

t 

0.04 

O 


Note, f Significant increase, <-> Change not significant, J Significant decrease 


• On average, students answered 65 percent of the items correctly. The top five 
differentially most difficult items all measured the cognitive target ‘Integrate and 
interpret information and ideas presented in text'. 
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DISTRICT PROFILES 


Austin: Grade 4 Mathematics 

• Scored no differently from the national and higher than the LC averages in overall 
mathematics in 2009. 

• Scored higher than 13 districts (all except Boston, Charlotte, and New York City) in 
overall mathematics in 2009 after adjusting for relevant background characteristics. No 
district scored higher than Austin in adjusted overall mathematics scores in 2009. 

• Average overall mathematics scale score corresponded to the 50 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
46 th (data) to 51 st (numbers). The average student was around the national median on all 
sub sc ales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 17. Austin's Changes in Grade 4 Mathematics Overall and Subscales Scores, 2007-2009 



National Public 

Large City 


Austin 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

-0.01 

O 

Numbers 

0.01 

O 

0.05 

t 

0.02 

O 

Measurement 

-0.01 

O 

0.07 

t 

-0.02 

O 

Geometry 

0.03 

t 

0.07 

t 

-0.03 

O 

Data 

-0.03 

t 

0.01 

O 

0.01 

O 

Algebra 

0.00 

O 

0.03 

O 

-0.08 

O 


Note, f Significant increase, <-> Change not significant, j, Significant decrease 


• On average, students answered 56 percent of the items correctly. The top five 
differentially most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths. 

o Graph or interpret points with whole number or letter coordinates on grids or in the 
first quadrant of the coordinate plane (two items), 
o Construct geometric figures with vertices at points on a coordinate grid, 
o Recognize or describe a relationship in which quantities change proportionally. 
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Austin: Grade 8 Mathematics 


• Scored higher than the national and LC averages in overall mathematics in 2009. 

• Scored higher than 16 (all except Boston) districts in overall mathematics in 2009 after 
adjusting for relevant background characteristics. No district scored higher than Austin in 
adjusted overall mathematics scores in 2009. 

• Average overall mathematics scale score corresponded to the 55 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
50 th (algebra) to 61th (geometry). The average student was at the national median in 
algebra and above the national median on all other subscales. 


• Displayed significant gain in overall mathematics and in the algebra subscale from 2007 
to 2009. 

Table 18. Austin's Changes in Grade 8 Mathematics Overall and Subscale Scores, 2007-2009 



National Public 

Large City 


Austin 



Effect Size 

Significance 

Effect Size 

Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 


t 

0.12 

t 

Numbers 

0.02 

t 

0.09 


t 

0.04 

O 

Measurement 

0.04 

t 

0.08 


t 

0.13 

o 

Geometry 

0.05 

t 

0.07 


t 

0.19 

o 

Data 

0.00 

O 

0.02 


O 

0.06 

o 

Algebra 

0.06 

t 

0.05 


O 

0.14 

t 


Note, f Significant increase, <-> Change not significant, Significant decrease 


• On average, students answered correctly 54 percent of the items correctly. The top five 
differentially most difficult items measured the following objectives: 

o Read or interpret data, including interpolating or extrapolating from data, 
o Identify lines of symmetry in plane figures or recognize and classify types of 
symmetries of plane figures. 

o Use place value to model and describe integers and decimals (two items), 
o Identify or represent functional relationships in meaningful contexts. 
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Baltimore City 


Baltimore City participated in grade 4 and grade 8 NAEP reading and mathematics assessments 
for the first time in 2009. It had the fourth lowest student- teacher ratio among the 18 TUDA 
districts. Fifty-nine percent of total expenditures were instructional. 

Table 19. Baltimore's Demographics, 2009 



Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Baltimore City: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than seven districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, and Atlanta) and higher than five districts (Milwaukee, Fresno, 
Philadelphia, Cleveland, and Detroit) in overall reading in 2009 after adjusting for 
relevant background characteristics. 

• Average scale scores corresponded to the 28 th percentile in both overall reading and the 
literary subscale, and the 27 th percentile in the information subscale on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• 2009 was the first year Baltimore City participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 45 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (two items), 
o Locate and recall information from text (two items). 

o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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Baltimore City: Grade 8 Reading 


• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than five districts (Miami-Dade County, Boston, Austin, Charlotte, and 
Houston) and higher than two districts (District of Columbia and Detroit) in overall 
reading in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 28 th percentile in both overall reading and the 
literary subscale, and the 29 th percentile in the information subscale on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• 2009 was the first year Baltimore City participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 56 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target ‘integrate and interpret information 
and ideas presented in text'. 
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DISTRICT PROFILES 


Baltimore City: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than six districts (Houston, Austin, Boston, Charlotte, New York City, and 
Miami-Dade County) and higher than nine districts (Philadelphia, Jefferson County, 
Chicago, District of Columbia, Los Angeles, Milwaukee, Cleveland, Fresno, and Detroit) 
in overall mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 27 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
the 22 nd (measurement) to the 34 th (data). The average student was below the first 
national quartile in measurement, around the first national quartile in numbers, and below 
the national median in geometry and data subscales. 

• 2009 was the first year Baltimore City participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 44 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Determine appropriate size of unit of measurement in problem situation involving 
such attributes as length, time, capacity, or weight, 
o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments (two items), 
o Solve problems involving perimeter of plane figures, 
o Order/compare whole numbers, decimals, or fractions. 
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Baltimore City: Grade 8 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than seven districts (Austin, Boston, Houston, Charlotte, Miami-Dade 
County, New York City, and San Diego) and higher than five districts (Jefferson County, 
District of Columbia, Milwaukee, Fresno, and Detroit) in overall mathematics in 2009 
after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 24 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
the 23 rd (algebra) to the 29 th (data). The average student was below first national quartile 
in algebra and around the first national quartile in all other subscales. 

• 2009 was the first year Baltimore City participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, the students answered 38 percent of items correctly. The top five 
differentially most difficult items measured the following objectives: 

o Select or use an appropriate type of unit for the attribute being measured such as 
length, area, angle, time, or volume. 

o Draw or sketch from a written description polygons, circles, or semicircles, 
o Determine the sample space for a given situation. 

o Demonstrate an understanding about the two- and three-dimensional shapes in our 
world through identifying, drawing, modeling, building, or taking apart, 
o Interpret probabilities within a given context. 



American Institutes for Research * Council of the Great City Schools • Spring 2011 


29 




Boston 


Boston participated in grade 4 and grade 8 NAEP reading and mathematics assessments in both 
2007 and 2009. It had the lowest student-teacher ratio among the 18 TUDA districts. Fifty-eight 
percent of total expenditures were instructional. 

Table 20. Boston's Demographics, 2009 


Number of Schools 

138 

Number of Students 

55,923 

Student/Teacher Ratio 

12.8 

Free and Reduced-Price Lunch 

74% 

Expenditures ($/student) 

Total 

20,324 

Instructional 

11,737 

Student and Staff Support 

3,440 

Administration 

1,464 

Operations, Food Service, and Other Support Staff 

3,682 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Boston: Grade 4 Reading 

• Scored lower than the national and higher than the LC averages in overall reading in 
2009. 

• Scored higher than 13 districts (all except Austin, Charlotte, Miami-Dade County, and 
New York City) in overall reading in 2009 after adjusting for relevant background 
characteristics. No district scored higher than Boston in adjusted overall reading scores in 
2009. 

• Average scale scores in overall reading and in the literary, and information subscales 
corresponded to the 40 th , 43 ld , and 38 th percentiles, respectively, on the national score 
distribution. The average student was below the national median on all three measures. 

• Displayed significant gain in overall reading and in the literary reading subscale from 
2007 to 2009. 


Table 21. Boston's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Boston 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.15 

t 

Information 

0.01 

o 

0.06 

t 

0.13 

O 

Literary 

-0.01 

o 

0.03 

O 

0.16 

t 


Note. \ Significant increase, <-> Change not significant, j Significant decrease 


• On average, students answered 53 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 



ADDENDUM TO PIECES OF THE PUZZLE: RECENT PERFORMANCE TRENDS IN URBAN DISTRICTS 


32 





Boston: Grade 8 Reading 


• Scored lower than the national and higher than the LC averages in overall reading in 
2009. 

• Scored higher than 14 districts (all except Austin, Atlanta, and Miami-Dade County) in 
overall reading in 2009 after adjusting for relevant background characteristics. No district 
scored higher than Boston in adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 41 st percentile in both overall reading and the 
information subscale, and the 42 nd percentile in literary subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed no significant gain in overall reading and reading subscales from 2007 to 2009. 


Table 22. Boston's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Boston 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.10 

O 

Information 

0.04 

t 

0.04 

O 

0.03 

O 

Literary 

0.02 

t 

0.05 

t 

0.14 

O 


Note. T Significant increase, <— > Change not significant, ( Significant decrease 


• On average, students answered 62 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target 'integrate and interpret information 
and ideas presented in text.' 
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DISTRICT PROFILES 


Boston: Grade 4 Mathematics 

• Scored lower than the national and higher than the LC averages in overall mathematics in 
2009. 

• Scored higher than 13 districts (all except Austin, Charlotte, Houston, and New York 
City) in overall mathematics in 2009 after adjusting for relevant background 
characteristics. No district scored higher than Boston in adjusted overall mathematics 
scores in 2009. 

• Average overall mathematics scale score corresponded to the 44 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
38 th (algebra and data) to 50 th (geometry). The average student was below the national 
median in all subscales except Geometry. 


• Displayed significant gain in overall mathematics and the geometry sub sc ale from 2007 
to 2009. 

Table 23. Boston's Changes in Grade 4 Mathematics Overall and Subscales Scores, 2007-2009 



National Public 

Large City 


Boston 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

0.13 

t 

Numbers 

0.01 

O 

0.05 

t 

0.11 

O 

Measurement 

-0.01 

O 

0.07 

t 

0.13 

O 

Geometry 

0.03 

t 

0.07 

t 

0.28 

t 

Data 

-0.03 

t 

0.01 

O 

0.04 

O 

Algebra 

0.00 

O 

0.03 

O 

0.03 

o 


Note, f Significant increase, «-» Change not significant, J, Significant decrease 


• On average, students answered 5 1 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths (two items). 

o Use informal probabilistic thinking to describe chance events, 
o Determine a simple probability from a context that includes a picture, 
o Determine appropriate size of unit of measurement in problem situation involving 
such attributes as length, time, capacity, or weight. 
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Boston: Grade 8 Mathematics 


• Scored no differently from the national and higher than the LC averages in overall 
mathematics in 2009. 

• Scored higher than 15 districts (all except Austin and Houston) in overall mathematics in 
2009 after adjusting for relevant background characteristics. No district scored higher 
than Boston in adjusted overall mathematics scores in 2009. 

• Average overall mathematics scale score corresponded to the 47 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
45 th (algebra and geometry) to 49 th (measurement and data). The average student was 
around the national median in all sub sc ales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 24. Boston's Changes in Grade 8 Mathematics Overall and Subscale Scores, 2007-2009 



National Public 

Large City 


Boston 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

0.08 

O 

Numbers 

0.02 

t 

0.09 

t 

0.08 

O 

Measurement 

0.04 

t 

0.08 

t 

0.15 

O 

Geometry 

0.05 

t 

0.07 

t 

0.03 

O 

Data 

0.00 

O 

0.02 

O 

0.12 

O 

Algebra 

0.06 

t 

0.05 

O 

0.03 

O 


Note, f Significant increase, <— > Change not significant, (, Significant decrease 


• On average, students answered 50 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Use place value to model and describe integers and decimals (two items), 
o Identify, define, or describe geometric shapes in the plane and in three-dimensional 
space given a visual representation. 

o Solve problems involving coordinate pairs on the rectangular coordinate system, 
o Identify or represent functional relationships in meaningful contexts. 
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Charlotte 


Charlotte participated in grade 4 and grade 8 NAEP reading and mathematics assessments in 
both 2007 and 2009. It had the seventh lowest student-teacher ratio among the 18 TUDA 
districts. Sixty percent of total expenditures were instructional. 

Table 25. Charlotte's Demographics, 2009 


Number of Schools 

168 

Number of Students 

135,064 

Student/Teacher Ratio 

14.5 

Free and Reduced-Price Lunch 

46% 

Expenditures ($/student) 

Total 

8,115 

Instructional 

5,045 

Student and Staff Support 

549 

Administration 

944 

Operations, Food Service, and Other Support Staff 

1,577 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Charlotte: Grade 4 Reading 

• Scored higher than the national and LC averages in overall reading in 2009. 

• Scored higher than 12 districts (Atlanta, San Diego, Jefferson County, Baltimore, 
Chicago, District of Columbia, Los Angeles, Milwaukee, Fresno, Philadelphia, 
Cleveland, and Detroit) in overall reading in 2009 after adjusting for relevant background 
characteristics. No district scored higher than Charlotte in adjusted overall reading scores 
in 2009. 

• Average scale scores corresponded to the 51 st percentile on the national score distribution 
for overall reading, and on the literary and information subscales. Charlotte was the only 
district where the average student was above the national median in all three measures in 
this assessment. 

• Displayed no significant change in overall reading or reading subscales from 2007 to 
2009. 


Table 26. Charlotte's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Charlotte 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.06 

O 

Information 

0.01 

o 

0.06 

t 

0.05 

O 

Literary 

-0.01 

o 

0.03 

O 

0.07 

O 


Note. 1 Significant increase, <-> Change not significant, J, Significant decrease 


• On average, students answered 59 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (four items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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Charlotte: Grade 8 Reading 


• Scored lower than the national and higher than the LC averages in overall reading in 
2009. 

• Scored higher than eight districts (Baltimore, Los Angeles, Cleveland, Jefferson County, 
Milwaukee, District of Columbia, Fresno, and Detroit) and lower than two districts 
(Miami-Dade County and Boston) in overall reading in 2009 after adjusting for relevant 
background characteristics. 

• Average scale scores corresponded to the 44 th percentile in both overall reading and the 
literary subscale, and the 43 rd percentile in the information subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 27. Charlotte's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Charlotte 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.00 

O 

Information 

0.04 

t 

0.04 

O 

0.04 

O 

Literary 

0.02 

t 

0.05 

t 

-0.06 

O 


Note. 1 Significant increase, <-> Change not significant, J Significant decrease 


• On average, students answered 62 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Critique and evaluate information and ideas in text and the ways in which authors 
present text (three items), 
o Locate and recall information from text, 
o Integrate and interpret information and ideas presented in text. 
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DISTRICT PROFILES 


Charlotte: Grade 4 Mathematics 

• Scored higher than the national and the LC averages in overall mathematics in 2009. 

• Scored higher than 13 districts (all except Austin, Boston, Houston, and New York City) 
in overall mathematics in 2009 after adjusting for relevant background characteristics. No 
district scored higher than Charlotte in adjusted overall mathematics scores in 2009. 

• Average overall mathematics scale score corresponded to the 56 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
49 th (measurement) to 60 th (data and algebra). The average student was above the national 
median in all subscales except measurement. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 28. Charlotte's Changes In Grade 4 Mathematics Overall and Subscales Scores, 
2007-2009 



National Public 

Large City 


Charlotte 



Effect Size 

Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

O 

0.05 


t 

0.04 


O 

Numbers 

0.01 

O 

0.05 


t 

0.08 


O 

Measurement 

-0.01 

o 

0.07 


t 

-0.02 


O 

Geometry 

0.03 

t 

0.07 


t 

-0.07 


O 

Data 

-0.03 

t 

0.01 


O 

0.15 


O 

Algebra 

0.00 

o 

0.03 


O 

0.00 


O 


Note. T Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 58 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths. 

o Identify or describe real-world objects using simple plane figures and simple solid 
figures. 

o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments (two items), 
o Solve problems involving perimeter of plane figures. 
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Charlotte: Grade 8 Mathematics 


• Scored no differently from the national and higher than the LC averages in overall 
mathematics in 2009. 

• Scored higher than 14 districts (all except Austin, Boston, and Houston) and lower than 
three districts (Austin, Boston and Houston) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 50 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
45 th (numbers) to 55 th (geometry). The average student was at the national median on 
data, above the national median on geometry, and below the national median in the other 
three subscales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 29. Charlotte's Changes in Grade 8 Mathematics Overall and Subscale Scores, 
2007-2009 



National Public 

Large City 


Charlotte 



Effect Size 

Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 


t 

-0.01 


O 

Numbers 

0.02 

t 

0.09 


t 

-0.01 


O 

Measurement 

0.04 

t 

0.08 


t 

0.04 


O 

Geometry 

0.05 

t 

0.07 


t 

0.07 


O 

Data 

0.00 

O 

0.02 


O 

0.02 


O 

Algebra 

0.06 

t 

0.05 


O 

-0.09 


O 


Note. | Significant increase, <-> Change not significant, [ Significant decrease 


• On average, students answered 51 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Explain or justify a mathematical concept or relationship, 
o Select or use an appropriate type of unit for the attribute being measured, 
o Compare objects with respect to length, area, volume, angle measurement, weight, or 
mass. 

o Solve problems involving conversions within the same measurement system, 
o Perform basic operations, using appropriate tools, on linear algebraic expressions. 
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Chicago 


Chicago participated in grade 4 and grade 8 NAEP reading and mathematics assessments in both 
2007 and 2009. It had the third highest student-teacher ratio among the 18 TUDA districts. Sixty 
percent of total expenditures were instructional. 

Table 30. Chicago's Demographics, 2009 


Number of Schools 

643 

Number of Students 

421,430 

Student/Teacher Ratio 

19.6 

Free and Reduced-Price Lunch 

73% 

Expenditures ($/student) 

Total 

10,392 

Instructional 

6,207 

Student and Staff Support 

1,381 

Administration 

966 

Operations, Food Service, and Other Support Staff 

1,838 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Chicago: Grade 4 Reading 

• Scored lower than the national and the LC average in overall reading in 2009. 

• Scored lower than eight districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, and San Diego) and higher than five districts (Milwaukee, 
Fresno, Philadelphia, Cleveland, and Detroit) in overall reading in 2009 after adjusting 
for relevant background characteristics. 

• Average scale scores corresponded to the 28 th percentile in both overall reading and the 
information subscale, and the 29 th percentile in the literary subscale on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 31. Chicago's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Chicago 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.05 

O 

Information 

0.01 

o 

0.06 

t 

0.07 

O 

Literary 

-0.01 

o 

0.03 

O 

0.02 

O 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 46 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets 

o Integrate and interpret information and ideas presented in text (two items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text (two items), 
o Locate and recall information from text. 
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Chicago: Grade 8 Reading 


• Scored lower than the national and no different from the LC averages in overall reading 
in 2009. 

• Scored lower than two districts (Miami-Dade County and Boston) and higher than five 
districts (Jefferson County, Milwaukee, District of Columbia, Fresno, and Detroit) in 
overall reading in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 32 nd percentile in both overall reading and the 
information subscale, and the 34 th percentile in the literary subscale on the national score 
distribution. The average student was below the national median on all three measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 32. Chicago's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Chicago 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

-0.01 

O 

Information 

0.04 

t 

0.04 

O 

-0.01 

O 

Literary 

0.02 

t 

0.05 

t 

-0.07 

O 


Note, f Significant increase, <— > Change not significant, ) Significant decrease 


• On average, students answered 58 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target 'integrate and interpret information 
and ideas presented in text.’ 
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DISTRICT PROFILES 


Chicago: Grade 4 Mathematics 

• Scored lower than the national and the LC averages in overall mathematics in 2009. 

• Scored lower than nine districts (Houston, Austin, Boston, Charlotte, New York City, 
Miami-Dade County, San Diego, Atlanta, and Baltimore) and higher than three districts 
(Cleveland, Fresno, and Detroit) in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 27 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
25 th (algebra) to 30 th (measurement and geometry). The average student was around the 
first national quartile in all subscales except measurement and geometry. 


• Displayed significant gains in the measurement and geometry subscales from 2007 to 
2009. 

Table 33. Chicago's Changes in Grade 4 Mathematics Overall and Subscales Scores, 2007-2009 



National Public 

Large City 


Chicago 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

0.08 

O 

Numbers 

0.01 

O 

0.05 

t 

0.05 

O 

Measurement 

-0.01 

O 

0.07 

t 

0.16 

t 

Geometry 

0.03 

t 

0.07 

t 

0.18 

t 

Data 

-0.03 

t 

0.01 

O 

-0.05 

O 

Algebra 

0.00 

O 

0.03 

O 

-0.03 

O 


Note, f Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 43 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths. 

o Identify place value and actual value of digits in whole numbers, 
o Read or interpret a single set of data. 

o Solve problems by estimating and computing within a single set of data, 
o Order/compare whole numbers, decimals, or fractions. 
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Chicago: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than five districts (Austin, Boston, Houston, Charlotte, and Miami-Dade 
County) and higher than six districts (Los Angeles, Jefferson County, District of 
Columbia, Milwaukee, Fresno, and Detroit) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 30 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
30 th (numbers and algebra) to 33 ld (measurement). The average student was below the 
national median in all subscales. 

• Displayed significant gains in measurement subscales from 2007 to 2009. 


Table 34. Chicago's Changes in Grade 8 Mathematics Overall and Subscale Scores, 2007-2009 



National Public 

Large City 


Chicago 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

0.09 

O 

Numbers 

0.02 

t 

0.09 

t 

0.03 

t 

Measurement 

0.04 

t 

0.08 

t 

0.16 

t 

Geometry 

0.05 

t 

0.07 

t 

0.06 

O 

Data 

0.00 

O 

0.02 

O 

0.03 

o 

Algebra 

0.06 

t 

0.05 

O 

0.12 

o 


Note, f Significant increase, <-> Change not significant, ) Significant decrease 


• On average, students answered 41 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Model or describe rational numbers or numerical relationships using number lines 
and diagrams. 

o Write algebraic expressions, equations, or inequalities to represent a situation, 
o Recognize or informally describe the effect of a transformation on two-dimensional 
geometric shapes. 

o Identify or represent functional relationships in meaningful contexts, 
o Interpret probabilities within a given context. 
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Cleveland 


Cleveland participated in grade 4 and grade 8 NAEP reading and mathematics assessments in 
both 2007 and 2009. It had the eighth lowest student-teacher ratio among the 18 TUDA districts. 
Sixty-five percent of total expenditures were instructional. 

Table 35. Cleveland's Demographics, 2009 


Number of Schools 

108 

Number of Students 

49,952 

Student/Teacher Ratio 

13.9 

Free and Reduced-Price Lunch 

100% 

Expenditures ($/student) 

Total 

12,393 

Instructional 

7,416 

Student and Staff Support 

1,552 

Administration 

1,392 

Operations, Food Service, and Other Support Staff 

2,033 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Cleveland: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 13 districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, San Diego, Jefferson County, Baltimore, Chicago, District 
of Columbia, and Los Angeles) in overall reading in 2009 after adjusting for relevant 
background characteristics. Cleveland did not score higher than any other district in 
adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 21 st percentile in both overall reading and the 
literary subscale, and the 22 nd percentile in the information subscale on the national score 
distribution. The average student was below the first national quartile in all three 
measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 36. Cleveland's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Cleveland 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

-0.14 

O 

Information 

0.01 

o 

0.06 

t 

-0.07 

O 

Literary 

-0.01 

o 

0.03 

O 

-0.20 

O 


Note. 1 Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 42 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Locate and recall information from text (three items), 
o Integrate and interpret information and ideas presented in text (two items). 
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Cleveland: Grade 8 Reading 


• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than six districts (Miami-Dade County, Boston, Austin, Atlanta, Charlotte, 
and Houston) and higher than two districts (District of Columbia and Detroit) in overall 
reading in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 26 th percentile in both overall reading and the 
information subscale, and the 27 th percentile in literary subscale on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• Displayed significant decrease in the literary subscale from 2007 to 2009. 


Table 37. Cleveland's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Cleveland 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

-0.13 

O 

Information 

0.04 

t 

0.04 

O 

-0.11 

O 

Literary 

0.02 

t 

0.05 

t 

-0.23 

T 


Note, f Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 56 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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DISTRICT PROFILES 


Cleveland: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than 14 districts (Houston, Austin, Boston, Charlotte, New York City, 
Miami-Dade County, San Diego, Atlanta, Baltimore, Philadelphia, Jefferson County, 
Chicago, District of Columbia, and Los Angeles) and higher than one district (Detroit) in 
overall mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 18 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
18 th (numbers) to 24 th (geometry). The average student was below the first national 
quartile in all subscales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 38. Cleveland's Changes in Grade 4 Mathematics Overall and Subscales Scores, 
2007-2009 



National Public 

Large City 


Cleveland 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

-0.07 

O 

Numbers 

0.01 

O 

0.05 

t 

-0.13 

O 

Measurement 

-0.01 

O 

0.07 

t 

-0.04 

O 

Geometry 

0.03 

t 

0.07 

t 

0.09 

o 

Data 

-0.03 

t 

0.01 

O 

-0.16 

o 

Algebra 

0.00 

O 

0.03 

O 

0.00 

o 


Note. ( Significant increase, <--> Change not significant, (, Significant decrease 


• On average, students answered 39 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Read or interpret a single set of data 
o Recognize, describe, or extend numerical patterns, 
o Use place value to model and describe integers and decimals, 
o For a given set of data, complete a graph. 

o Represent numbers using models such as base 10 representations, number lines and 
two-dimensional models. 
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Cleveland: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than seven districts (Austin, Boston, Houston, Charlotte, Miami-Dade 
County, New York City, and San Diego) and higher than six districts (Los Angeles, 
Jefferson County, District of Columbia, Milwaukee, Fresno, and Detroit) in overall 
mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 23 percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
23 ld (measurement and data) to the 26 th (numbers and geometry). The average student 
was below the first national quartile in all subscales except numbers and geometry. 

• Displayed no significant change in overall mathematics or mathematics subscales from 
2007 to 2009. 

Table 39. Cleveland's Changes in Grade 8 Mathematics Overall and Subscale Scores, 

2007-2009 



National Public 

Large City 


Cleveland 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

-0.02 

O 

Numbers 

0.02 

t 

0.09 

t 

0.14 

O 

Measurement 

0.04 

t 

0.08 

t 

-0.06 

O 

Geometry 

0.05 

t 

0.07 

t 

-0.03 

O 

Data 

0.00 

O 

0.02 

O 

-0.08 

O 

Algebra 

0.06 

t 

0.05 

o 

-0.08 

O 


Note. \ Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 37 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Describe relative positions of points and lines using the geometric ideas of midpoint, 
points on common line through a common point, parallelism, or perpendicularity, 
o Model or describe rational numbers or numerical relationships using number lines 
and diagrams (two items). 

o Solve problems involving conversions within the same measurement system, 
o Interpret probabilities within a given context. 
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Detroit 


Detroit participated in grade 4 and grade 8 NAEP reading and mathematics assessments for the 
first time in 2009. It had the sixth highest student-teacher ratio among the 18 TUDA districts. 
Fifty-four percent of total expenditures were instructional. 

Table 40. Detroit's Demographics, 2009 


Number of Schools 

199 

Number of Students 

97,577 

Student/Teacher Ratio 

16.4 

Free and Reduced-Price Lunch 

77% 

Expenditures ($/student) 

Total 

12,016 

Instructional 

6,522 

Student and Staff Support 

1,378 

Administration 

1,535 

Operations, Food Service, and Other Support Staff 

2,581 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Detroit: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 14 districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, San Diego, Jefferson County, Baltimore, Chicago, District 
of Columbia, Los Angeles, and Fresno) in overall reading in 2009 after adjusting for 
relevant background characteristics. Detroit did not score higher than any other district in 
adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 16 th percentile in both overall reading and the 
information subscale, and the 18 th percentile in the literary subscale on the national score 
distribution. The average student was below the first national quartile in all three 
measures. 

• 2009 was the first year Detroit participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 38 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (two items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text (two items), 
o Locate and recall information from text. 
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Detroit: Grade 8 Reading 


• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 15 districts (Miami-Dade County, Boston, Austin, Atlanta, Charlotte, 
Houston, Chicago, New York City, San Diego, Philadelphia, Baltimore, Los Angeles, 
Cleveland, Jefferson County, and Milwaukee) in overall reading in 2009 after adjusting 
for relevant background characteristics. Detroit did not score higher than any other 
district in adjusted overall reading scores in 2009. 

• Average scale scores in overall reading, and the literary and information subscales 
corresponded to the 18 th , 21 st , and 17 th percentiles, respectively, on the national score 
distribution. The average student was below the first national quartile in all three 
measures. 

• 2009 was the first year Detroit participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 49 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target 'integrate and interpret information 
and ideas presented in text.' 
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DISTRICT PROFILES 


Detroit: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than all other districts in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 9 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 9 th 
(numbers) to 13 th (geometry). The average student was below the first national quartile in 
all subscales. 

• 2009 was the first year Detroit participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 32 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths. 

o Use place value to model and describe integers and decimals, 
o Compose or decompose whole quantities by place value. 

o Select or use appropriate measurement instruments such as ruler, meter stick clock, 
thermometer, or other scaled instruments, 
o Order/compare whole numbers, decimals, or fractions. 
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Detroit: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than all other districts in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 12 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
12 th (measurement, geometry and data) to 14 th (numbers and algebra). The average 
student was below the first national quartile in all subscales. 

• 2009 was the first year Detroit participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 30 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Analyze a situation that involves probability of an independent event, 
o Use place value to model and describe integers and decimals. 

o Compare objects with respect to length, area, volume, angle measurement, weight, or 
mass. 

o Solve problems involving conversions within the same measurement system, 
o Perform computations with rational numbers. 
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District of Columbia 12 


District of Columbia participated in grade 4 and grade 8 NAEP reading and mathematics 
assessments in both 2007 and 2009. It had the second lowest student-teacher ratio among the 18 


TUDA districts. Forty-five percent of total expenditures were instructional. 

Table 41. District of Columbia's Demographics, 2009 


Number of Schools 

172 

Number of Students 

44,331 

Student/Teacher Ratio 

12.5 

Free and Reduced-Price Lunch 
Expenditures ($/student) 

69% 

Total 

14,594 

Instructional 

6,542 

Student and Staff Support 

2,069 

Administration 

2,359 

Operations, Food Service, and Other Support Staff 

3,625 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 

12 This report includes charters in the TUDA analysis according to the rules that were in place in 2007 and 2009. 
Beginning in 2009, TUDA samples included only those charter schools that each district included for the purpose of 
reporting Adequate Yearly Progress to the US Department of Education under the Elementary and Secondary 
Education Act. This rule did not exist in 2007, so the TUDA sample that year included charters without this AYP 
distinction. The inclusion or exclusion of charter schools from the TUDA samples did not affect the significance of 
the change in reported scores between 2007 and 2009, with the exception of the District of Columbia. Therefore, we 
removed charters from the District of Columbia sample in both years in order to ensure that the scores in 2007 and 
2009 for DCPS were comparable. 
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DISTRICT PROFILES 


District of Columbia: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored higher than five districts (Milwaukee, Fresno, Philadelphia, Cleveland and 
Detroit) and lower than nine districts (Boston, Miami-Dade County, New York City, 
Austin, Charlotte, Houston, Atlanta, San Diego, and Jefferson County) in overall reading 
in 2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading, and the literary and information subscales 
corresponded to the 29 th , 31 st , and 28 th percentiles, respectively, on the national score 
distribution. The average student was around the first national quartile in overall reading 
and literary subscale and below the national median in the information subscale. 

• Displayed significant gains in overall reading and both reading subscales from 2007 to 
2009. Changes in average scores were statistically no different from those of the national 
and large city populations. 


Table 42. District of Columbia's Changes in Grade 4 Reading Overall and Subscale Scores, 
2007-2009 


National Public 


Large City 


District of Columbia 

Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.15 

t 

Information 

0.01 

o 

0.06 

t 

0.13 

t 

Literary 

-0.01 

o 

0.03 

O 

0.15 

t 


Note, f Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 47 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Locate and recall information from text (four items), 
o Integrate and interpret information and ideas presented in text. 
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District of Columbia: Grade 8 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 14 districts (Miami-Dade County, Boston, Austin, Atlanta, Charlotte, 
Houston, Chicago, New York City, San Diego, Philadelphia, Baltimore, Los Angeles, 
Cleveland, and Jefferson County) in overall reading in 2009 after adjusting for relevant 
background characteristics. 

• Average scale scores corresponded to the 24 th percentile in overall reading and to the 25 th 
percentile in the literary and information subscales on the national score distribution. The 
average student was around the first national quartile in all three measures. 

• Displayed significant decrease in the literary subscale from 2007 to 2009. Changes in 
average scores were statistically no different from those of the national and large city 
populations 

Table 43. District of Columbia's Changes in Grade 8 Reading Overall and Subscale Scores, 
2007-2009 


National Public 


Large City 


District of Columbia 

Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.10 

t 

Information 

0.04 

t 

0.04 

O 

0.11 

O 

Literary 

0.02 

t 

0.05 

t 

0.00 

O 


Note. 1 Significant increase, <-> Change not significant, l Significant decrease 


• On average, students answered 55 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (four items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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DISTRICT PROFILES 


District of Columbia: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored higher than three districts (Cleveland, Fresno, and Detroit) and lower than nine 
districts (Houston, Austin, Boston, Charlotte, New York City, Miami-Dade County, San 
Diego, Atlanta, and Baltimore) in overall mathematics in 2009 after adjusting for relevant 
background characteristics. 

• Average overall mathematics scale score corresponded to the 24 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
24 th (measurement) to 29 th (algebra). The average student was around the first national 
quartile in all subscales. 

• Displayed significant gains in overall mathematics and all five mathematics subscales 
from 2007 to 2009. Changes in average scores were statistically higher than that of the 
national population in algebra. 

Table 44. District of Columbia's Changes in Grade 4 Mathematics Overall and Subscales 
Scores, 2007-2009 



National Public 

Large City 


District of Columbia 


Effect Size 

Significance 

Effect Size Significance 

Effect Size 

Significance 

Composite 

0.00 

O 

0.05 

t 

0.19 

t 

Numbers 

0.01 

O 

0.05 

t 

0.13 

t 

Measurement 

-0.01 

o 

0.07 

t 

0.19 

t 

Geometry 

0.03 

t 

0.07 

t 

0.14 

t 

Data 

-0.03 

t 

0.01 

O 

0.24 

t 

Algebra 

0.00 

o 

0.03 

O 

0.28 

t 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 43 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Identify odd and even numbers, 
o Read or interpret a single set of data. 

o Graph or interpret points with whole number or letter coordinates on grids or in the 
first quadrant of the coordinate plane, 
o Recognize two-dimensional faces of three-dimensional shapes. 

o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments. 
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District of Columbia: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored higher than one district (Detroit) and lower than 12 districts (Austin, Boston, 
Houston, Charlotte, Miami-Dade County, New York City, San Diego, Chicago, 
Philadelphia, Atlanta, Cleveland, and Baltimore) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 20 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
19 th (measurement and geometry) to 24 th (data). The average student was below the first 
national quartile in all subscales. 

• Displayed significant gains in overall mathematics and all subscales except data from 
2007 to 2009. 

Table 45. District of Columbia's Changes in Grade 8 Mathematics Overall and Subscale Scores, 
2007-2009 



National Public 

Large City 


District of Columbia 


Effect Size 

Significance 

Effect Size Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.19 

t 

Numbers 

0.02 

t 

0.09 

t 

0.20 

t 

Measurement 

0.04 

t 

0.08 

t 

0.22 

t 

Geometry 

0.05 

t 

0.07 

t 

0.22 

t 

Data 

0.00 

O 

0.02 

O 

0.14 

O 

Algebra 

0.06 

t 

0.05 

O 

0.13 

t 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 36 percent of items correctly. The top five differentially 

most difficult items measured the following objectives: 

o Select or use an appropriate type of unit for the attribute being measured such as 
length, area, angle, time, or volume. 

o Model or describe rational numbers or numerical relationships using number lines 
and diagrams. 

o Recognize or informally describe the effect of a transformation on two-dimensional 
geometric shapes. 

o Draw or sketch from a written description polygons, circles, or semicircles. 

o Construct or solve problems involving scale drawings. 
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Fresno 


Fresno participated in grade 4 and grade 8 NAEP reading and mathematics assessments for the 
first time in 2009. It had the third highest student-teacher ratio among the 18 TUDA districts. 
Sixty percent of total expenditures were instructional. 

Table 46. Fresno's Demographics, 2009 


Number of Schools 

107 

Number of Students 

76,621 

Student/Teacher Ratio 

19.5 

Free and Reduced-Price Lunch 

79% 

Expenditures ($/student) 

Total 

10,053 

Instructional 

5,990 

Student and Staff Support 

1,420 

Administration 

859 

Operations, Food Service, and Other Support Staff 

1,784 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Fresno: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 13 districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, San Diego, Jefferson County, Baltimore, Chicago, District 
of Columbia, and Los Angeles) and higher than one district (Detroit) in overall reading in 
2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading, and the literary and information subscales 
corresponded to the 23 rd , 27 th , and 21 st percentiles, respectively, on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• 2009 was the first year Fresno participated in NAEP TUDA. Thus, there are no trend data 
for this district. 

• On average, students answered 43 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target 'integrate and interpret information 
and ideas presented in text.' 
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Fresno: Grade 8 Reading 


• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than nine districts (Miami-Dade County, Boston, Austin, Atlanta, Charlotte, 
Houston, Chicago, New York City, and Los Angeles) in overall reading in 2009 after 
adjusting for relevant background characteristics. Fresno did not score higher than any 
other district in adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 23 percentile in both overall reading and the 
information subscales and to the 25 th percentile in the literary subscale on the national 
score distribution. The average student was around the first national quartile in all three 
measures. 

• 2009 was the first year Fresno participated in NAEP TUDA. Thus, there are no trend data 
for this district. 

• On average, students answered 52 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Locate and recall information from text (three items), 
o Integrate and interpret information and ideas presented in text. 

o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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DISTRICT PROFILES 


Fresno: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than 15 districts (Houston, Austin, Boston, Charlotte, New York City, 
Miami-Dade County, San Diego, Atlanta, Baltimore, Philadelphia, Jefferson County, 
Chicago, District of Columbia, Los Angeles, and Milwaukee) and higher than one district 
(Detroit) in overall mathematics in 2009 after adjusting for relevant background 
characteristics. 

• Average overall mathematics scale score corresponded to the 23 percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
19 th (measurement) to 29 th (numbers). The average student was below the first national 
quartile in all subscales except numbers. 

• 2009 was the first year Fresno participated in NAEP TUDA. Thus, there are no trend data 
for this district. 

• On average, students answered 43 percents of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o For a given set of data, complete a graph. 

o Graph or interpret points with whole number or letter coordinates on grids or in the 
first quadrant of the coordinate plane. 

o Solve problems by estimating and computing within a single set of data, 
o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments (two items). 



ADDENDUM TO PIECES OF THE PUZZLE: RECENT PERFORMANCE TRENDS IN URBAN DISTRICTS 


70 




Fresno: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than 13 districts (Austin, Boston, Houston, Charlotte, Miami-Dade County, 
New York City, San Diego, Chicago, Philadelphia, Atlanta, Cleveland, Baltimore, and 
Los Angeles) and higher than one district (Detroit) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 25 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
23 ld (data) to 31 st (numbers). The average student was below the national median in 
numbers, below the first national quartile in geometry and data, and around the first 
national quartile in the other three subscales. 

• 2009 was the first year Fresno participated in NAEP TUDA. Thus, there are no trend data 
for this district. 

• On average, students answered 39 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Read or interpret data, including interpolating or extrapolating from data, 
o Determine the probability of independent and dependent events, 
o Recognize or informally describe the effect of a transformation on two-dimensional 
geometric shapes (2 items), 
o Interpret probabilities within a given context. 
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Houston 


Houston participated in grade 4 and grade 8 NAEP reading and mathematics assessments in both 
2007 and 2009. It had the seventh highest student-teacher ratio among the 18 TUDA districts. 
Fifty-nine percent of total expenditures were instructional. 

Table 47. Houston's Demographics, 2009 


Number of Schools 

305 

Number of Students 

200,225 

Student/Teacher Ratio 

16.7 

Free and Reduced-Price Lunch 

63% 

Expenditures ($/student) 

Total 

8,604 

Instructional 

5,048 

Student and Staff Support 

853 

Administration 

944 

Operations, Food Service, and Other Support Staff 

1,758 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Houston: Grade 4 Reading 

• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored higher than 11 districts (San Diego, Jefferson County, Baltimore, Chicago, 
District of Columbia, Los Angeles, Milwaukee, Fresno, Philadelphia, Cleveland, and 
Detroit) and lower than two districts (Boston and Miami-Dade County) in overall reading 
in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 36 th percentile in both overall reading and the 
information subscale, and to the 38 th percentile in literary subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed significant gain in overall reading and in the two reading subscales from 2007 
to 2009. 


Table 48. Houston's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Houston 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.17 

t 

Information 

0.01 

o 

0.06 

t 

0.15 

t 

Literary 

-0.01 

o 

0.03 

O 

0.18 

t 


Note, f Significant increase, <— > Change not significant, [ Significant decrease 


• On average, students answered 5 1 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target 'integrate and interpret information 
and ideas presented in text.' 
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Houston: Grade 8 Reading 


• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored higher than eight districts (Baltimore, Los Angeles, Cleveland, Jefferson County, 
Milwaukee, District of Columbia, Fresno, and Detroit) and lower than two districts 
(Miami-Dade County and Boston) in overall reading in 2009 after adjusting for relevant 
background characteristics. 

• Average scale scores corresponded to the 35 th percentile in both overall reading and the 
information subscale, and the 36 th percentile in the literary subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed no significant change in overall reading and reading subscales from 2007 to 
2009. 


Table 49. Houston's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Houston 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.00 

O 

Information 

0.04 

t 

0.04 

O 

0.04 

O 

Literary 

0.02 

t 

0.05 

t 

-0.05 

O 


Note, f Significant increase, <-> Change not significant, [ Significant decrease 


• On average, students answered 60 percent of items correctly. The top five differentially 
most difficult items all measured the cognitive target 'integrate and interpret information 
and ideas presented in text.' 
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DISTRICT PROFILES 


Houston: Grade 4 Mathematics 

• Scored lower than the national and higher than the LC averages in overall mathematics in 
2009. 

• Scored higher than 13 districts (all except Austin, Boston, Charlotte, and New York City) 
in overall mathematics in 2009 after adjusting for relevant background characteristics. No 
district scored higher than Houston in adjusted overall mathematics scores in 2009. 

• Average overall mathematics scale score corresponded to the 44 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
38 th (geometry) to 47 th (measurement). The average student was around the national 
median on measurement and below the national median in all other subscales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 

Table 50. Houston's Changes in Grade 4 Mathematics Overall and Subscales Scores, 2007-2009 



National Public 

Large City 


Houston 



Effect Size 

Significance 

Effect Size Significance Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

0.06 

O 

Numbers 

0.01 

O 

0.05 

t 

0.11 

O 

Measurement 

-0.01 

O 

0.07 

t 

0.12 

O 

Geometry 

0.03 

t 

0.07 

t 

-0.03 

O 

Data 

-0.03 

t 

0.01 

O 

-0.02 

O 

Algebra 

0.00 

O 

0.03 

o 

-0.03 

O 


Note, f Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 52 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Explain or justify a mathematical concept or relationship, 
o Identify odd and even numbers, 
o Read or interpret a single set of data. 

o Construct geometric figures with vertices at points on a coordinate grid, 
o Determine appropriate size of unit of measurement in problem situation involving 
such attributes as length, time, capacity, or weight. 
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Houston: Grade 8 Mathematics 


• Scored lower than the national and higher than the LC averages in overall mathematics in 
2009. 

• Scored higher than 15 districts (all except Austin and Boston) and lower than one district 
(Austin) in overall mathematics in 2009 after adjusting for relevant background 
characteristics. 

• Average overall mathematics scale score corresponded to the 43 percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
40 th (algebra) to 47 th (measurement). The average student was around the national median 
in measurement, numbers and geometry and below the national median in the other two 
subscales. 


• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 

Table 51. Houston's Changes in Grade 8 Mathematics Overall and Subscale Scores, 2007-2009 



National Public 

Large City 

Houston 



Effect Size 

Significance 

Effect Size 

Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

0.10 

O 

Numbers 

0.02 

t 

0.09 

t 

0.12 

O 

Measurement 

0.04 

t 

0.08 

t 

0.13 

O 

Geometry 

0.05 

t 

0.07 

t 

0.02 

O 

Data 

0.00 

O 

0.02 

O 

0.09 

O 

Algebra 

0.06 

t 

0.05 

O 

0.10 

O 


Note. \ Significant increase, <-> Change not significant, j, Significant decrease 


• On average, students answered 48 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Solve linear equations or inequalities. 

o Perform basic operations, using appropriate tools, on linear algebraic expressions, 
o Visually choose the line that best fits given a scatterplot and informally explain the 
meaning of the line. Use the line to make predictions, 
o Identify functions as linear or nonlinear or contrast distinguishing properties of 
functions from tables, graphs, or equations, 
o Identify or represent functional relationships in meaningful contexts. 
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Jefferson County 


Jefferson County participated in grade 4 and grade 8 NAEP reading and mathematics for the first 
time in 2009. It had the ninth lowest student-teacher ratio among the 18 TUDA districts. Fifty- 
four percent of total expenditures were instructional. 

Table 52. Jefferson County's Demographics, 2009 


Number of Schools 

174 

Number of Students 

98,774 

Student/Teacher Ratio 

16.1 

Free and Reduced-Price Lunch 

56% 

Expenditures ($/student) 

Total 

9,966 

Instructional 

5,350 

Student and Staff Support 

1,401 

Administration 

1,144 

Operations, Food Service and Other Support Staff 

2,072 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Jefferson County: Grade 4 Reading 

• Scored no differently from the national and higher than the LC average in overall reading 
in 2009. 

• Scored lower than six districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, and Houston) and higher than seven districts (District of Columbia, Los 
Angeles, Milwaukee, Fresno, Philadelphia, Cleveland, and Detroit) in overall reading in 
2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading, and in the literary and information subscales 
corresponded to the 45 th , 44 th , and 47 th percentiles, respectively, on the national score 
distribution. The average student was close to the national median in all three measures. 

• 2009 was the first year Jefferson County participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 54 percent of items correctly. The five differentially most 
difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (four items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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Jefferson County: Grade 8 Reading 


• Scored lower than the national and higher than the LC averages in overall reading in 
2009. 

• Scored lower than seven districts (Miami-Dade County, Boston, Austin, Atlanta, 
Charlotte, Houston, and Chicago) and higher than two districts (District of Columbia and 
Detroit) in overall reading in 2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading, in the literary and information subscales 
corresponded to the 42 nd , 40 th , and 45 th percentiles, respectively, on the national score 
distribution. The average student was below the national median in all three measures. 

• 2009 was the first year Jefferson County participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 64 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text. 

o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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DISTRICT PROFILES 


Jefferson County: Grade 4 Mathematics 

• Scored lower than the national and no differently from the LC averages in overall 
mathematics in 2009. 

• Scored lower than nine districts (Houston, Austin, Boston, Charlotte, New York City, 
Miami-Dade County, San Diego, Atlanta, and Baltimore) and higher than three districts 
(Cleveland, Fresno, and Detroit) in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 40 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
36 th (numbers) to 48 th (geometry). The average student was around the national median in 
geometry and below the national median in all other subscales. 

• 2009 was the first year Jefferson County participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 50 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths (three items), 
o Multiply whole numbers, 
o Divide whole numbers. 
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Jefferson County: Grade 8 Mathematics 

• Scored lower than the national and no differently from the LC averages in overall 
mathematics in 2009. 

• Scored lower than 12 districts (Austin, Boston, Houston, Charlotte, Miami-Dade County, 
New York City, San Diego, Chicago, Philadelphia, Atlanta, Cleveland, and Baltimore) 
and higher than one district (Detroit) in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 38 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
36 th (numbers) to 39 th (geometry and algebra). The average student was below the 
national median in all sub sc ales. 

• 2009 was the first year Jefferson County participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 44 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Calculate, use, or interpret mean, median, mode, or range, 
o Perform computations with rational numbers. 

o Recognize, describe, or extend numerical and geometric patterns using tables, graphs, 
words, or symbols. 

o Select or use an appropriate type of unit for the attribute being measured such as 
length, area, angle, time, or volume. 

o Interpret relationships between symbolic linear expressions and graphs of lines by 
identifying and computing slope and intercepts. 
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Los Angeles 


Los Angeles participated in grade 4 and grade 8 NAEP reading and mathematics assessments in 
both 2007 and 2009. It had the highest student-teacher ratio among the 18 TUDA districts. Fifty- 
nine percent of total expenditures were instructional. 

Table 53. Los Angeles' Demographics, 2009 


Number of Schools 

868 

Number of Students 

687,534 

Student/Teacher Ratio 

19.6 

Free and Reduced-Price Lunch 

75% 

Expenditures ($/student) 

Total 

11,357 

Instructional 

6,666 

Student and Staff Support 

1,619 

Administration 

1,238 

Operations, Food Service, and Other Support Staff 

1,834 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Los Angeles: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than nine districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, San Diego, and Jefferson County) and higher than five 
districts (Milwaukee, Fresno, Philadelphia, Cleveland, and Detroit) in overall reading in 
2009 after adjusting for relevant background characteristics. 

• Average scale scores in overall reading, and in literary and information subscales 
corresponded to the 24 th , 25 th , and 23 rd percentiles, respectively, on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• Displayed no significant change in overall reading or in the reading subscales from 2007 
to 2009. 


Table 54. Los Angeles' Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Los Angeles 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.04 

O 

Information 

0.01 

o 

0.06 

t 

0.03 

O 

Literary 

-0.01 

o 

0.03 

O 

0.05 

O 


Note. 1 Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 44 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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Los Angeles: Grade 8 Reading 


• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than six districts (Miami-Dade County, Boston, Austin, Atlanta, Charlotte, 
and Houston) and higher than three districts (District of Columbia, Fresno, and Detroit) 
in overall reading in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 27 th percentile in both overall reading and 
information subscale and to the 28 th percentile in literary subscale on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• Displayed significant gain in overall reading from 2007 to 2009. 


Table 55. Los Angeles' Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


Los Angeles 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.09 

t 

Information 

0.04 

t 

0.04 

O 

0.08 

O 

Literary 

0.02 

t 

0.05 

t 

0.03 

O 


Note. ) Significant increase, <— > Change not significant, (, Significant decrease 


• On average, students answered 55 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Locate and recall information from text (three items), 
o Integrate and interpret information and ideas presented in text (two items). 



American Institutes for Research * Council of the Great City Schools • Spring 2011 


87 




DISTRICT PROFILES 


Los Angeles: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than nine districts (Houston, Austin, Boston, Charlotte, New York City, 
Miami-Dade County, San Diego, Atlanta, and Baltimore) and higher than three districts 
(Cleveland, Fresno, and Detroit) in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 26 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
23 ld (measurement) to 30 th (numbers). The average student was above the first national 
quartile in numbers and around the first national quartile in all other subscales. 

• Displayed no significant change in overall mathematics and mathematics subscales from 
2007 to 2009. 


Table 56. Los Angles' Changes in Grade 4 Mathematics Overall and Subscales Scores, 
2007-2009 



National Public 

Large City 


Los Angeles 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

0.03 

O 

Numbers 

0.01 

O 

0.05 

t 

0.04 

O 

Measurement 

-0.01 

O 

0.07 

t 

0.05 

O 

Geometry 

0.03 

t 

0.07 

t 

-0.01 

O 

Data 

-0.03 

t 

0.01 

O 

0.00 

O 

Algebra 

0.00 

O 

0.03 

o 

0.03 

O 


Note, f Significant increase, <-> Change not significant, (, Significant decrease 


• On average, students answered 44 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Use place value to model and describe integers and decimals, 
o For a given set of data, complete a graph. 

o Identify the images resulting from flips (reflections), slides (translations), or turns 
(rotations). 

o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments, 
o Solve application problems involving numbers and operations. 
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Los Angeles: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than 1 1 districts (Austin, Boston, Houston, Charlotte, Miami-Dade County, 
New York City, San Diego, Chicago, Philadelphia, Atlanta, and Cleveland) and higher 
than three districts (Milwaukee, Fresno, and Detroit) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 25 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
23 ld (measurement) to 30 th (numbers). The average student was below the national 
median on numbers and around the first national quartile in geometry and algebra and 
below the first national quartile in the other two subscales. 

• Displayed significant gains in the numbers subscale from 2007 to 2009. 


Table 57. Los Angeles' Changes in Grade 8 Mathematics Overall and Subscale Scores, 
2007-2009 



National Public 

Large City 


Los Angeles 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

0.03 

O 

Numbers 

0.02 

t 

0.09 

t 

0.14 

t 

Measurement 

0.04 

t 

0.08 

t 

0.08 

O 

Geometry 

0.05 

t 

0.07 

t 

-0.10 

O 

Data 

0.00 

O 

0.02 

O 

0.01 

O 

Algebra 

0.06 

t 

0.05 

o 

0.01 

O 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 39 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Read or interpret data, including interpolating or extrapolating from data, 
o Identify lines of symmetry in plane figures or recognize and classify types of 
symmetries of plane figures. 

o Determine the probability of independent and dependent events, 
o Draw or sketch from a written description polygons, circles, or semicircles, 
o Select or use an appropriate type of unit for the attribute being measured such as 
length, area, angle, time, or volume. 


■ 
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Miami-Dade County 


Miami-Dade County participated in grade 4 and grade 8 NAEP reading and mathematics 
assessments for the first time in 2009. It had the ninth lowest student-teacher ratio among the 18 
TUDA districts. Sixty-one percent of total expenditures were instructional. 

Table 58. Miami-Dade County’s Demographics, 2009 


Number of Schools 

557 

Number of Students 

345,525 

Student/Teacher Ratio 

15.4 

Free and Reduced-Price Lunch 

63% 

Expenditures ($/student) 

Total 

9,933 

Instructional 

6,057 

Student and Staff Support 

1,070 

Administration 

880 

Operations, Food Service, and Other Support Staff 

1,927 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Miami-Dade County: Grade 4 Reading 

• Scored no differently from the national and higher than the LC average in overall reading 
in 2009. 

• Scored higher than 13 districts (Houston, Atlanta, San Diego, Jefferson County, 
Baltimore, Chicago, District of Columbia, Los Angeles, Milwaukee, Fresno, 
Philadelphia, Cleveland, and Detroit) in overall reading in 2009 after adjusting for 
relevant background characteristics. No district scored higher than Miami-Dade County 
in adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 47 th percentile in both overall reading and in 
the information subscale, and to the 48 th percentile in literary subscale on the national 
score distribution. The average student was close to the national median in all three 
measures. 

• 2009 was the first year Miami-Dade County participated in NAEP TUDA. Thus, there 
are no trend data for this district. 

• On average, students answered 55 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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Miami-Dade County: Grade 8 Reading 


• Scored no differently from the national and higher than the LC averages in overall 
reading in 2009. 

• Scored higher than 15 districts (Atlanta, Charlotte, Houston, Chicago, New York City, 
San Diego, Philadelphia, Baltimore, Los Angeles, Cleveland, Jefferson County, 
Milwaukee, District of Columbia, Fresno, and Detroit) in overall reading in 2009 after 
adjusting for relevant background characteristics. No district scored higher than Miami- 
Dade County in adjusted overall reading scores in 2009. 

• Average scale scores in overall reading, and in the literary and information subscales in 
2009 corresponded to the 45 th , 46 th , and 44 th percentiles, respectively, on the national 
score distribution. The average student was close to the national median in these three 
measures. 

• 2009 was the first year Miami-Dade County participated in NAEP TUDA. Thus, there 
are no trend data for this district. 

• On average, students answered 63 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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DISTRICT PROFILES 


Miami-Dade County: Grade 4 Mathematics 

• Scored lower than the national and higher than the LC averages in overall mathematics in 
2009. 

• Scored higher than 11 districts (Atlanta, Baltimore, Philadelphia, Jefferson County, 
Chicago, District of Columbia, Los Angeles, Milwaukee, Cleveland, Fresno, and Detroit) 
and lower than five districts (Houston, Austin, Boston, Charlotte, and New York City) in 
overall mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 44 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
41 st (numbers) to 51 st (data). The average student was below the national median in 
numbers and measurement, around the national median in geometry and algebra, and 
slightly above the national median in the data subscale. 

• 2009 was the first year Miami-Dade County participated in NAEP TUDA. Thus, there 
are no trend data for this district. 

• On average, students answered 53 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths. 

o Explain or justify a mathematical concept or relationship, 
o Assemble simple plane shapes to construct a given shape, 
o Describe the effect of operations on size (whole numbers), 
o Solve application problems involving numbers and operations. 
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Miami-Dade County: Grade 8 Mathematics 

• Scored lower than the national and no differently from the LC averages in overall 
mathematics in 2009. 

• Scored higher than 11 districts (Chicago, Philadelphia, Atlanta, Cleveland, Baltimore, 
Los Angeles, Jefferson County, District of Columbia, Milwaukee, Fresno, and Detroit) 
and lower than four districts (Austin, Boston, Houston, and Charlotte) in overall 
mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 40 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
36 th (numbers) to 41 st (measurement). The average student was below the national 
median in all subscales. 

• 2009 was the first year Miami-Dade County participated in NAEP TUDA. Thus, there 
are no trend data for this district. 

• On average, students answered 45 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Describe relative positions of points and lines using the geometric ideas of midpoint, 
points on common line through a common point, parallelism, or perpendicularity, 
o Perform computations with rational numbers. 

o Identify, define, or describe geometric shapes in the plane and in three-dimensional 
space given a visual representation. 

o Perform basic operations, using appropriate tools, on linear algebraic expressions, 
o Construct or solve problems involving scale drawings. 
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Milwaukee 


Milwaukee participated in grade 4 and grade 8 NAEP reading and mathematics assessments for 
the first time in 2009. It had the fifth highest student-teacher ratio among the 18 TUDA districts. 
Fifty-seven percent of total expenditures were instructional. 

Table 59. Milwaukee's Demographics, 2009 


Number of Schools 

220 

Number of Students 

85,381 

Student/Teacher Ratio 

16.6 

Free and Reduced-Price Lunch 

77% 

Expenditures ($/student) 

Total 

12,705 

Instructional 

7,242 

Student and Staff Support 

1,430 

Administration 

1,840 

Operations, Food Service and Other Support Staff 

2,193 


Source: Common Core of Data public school district data for the 2008-2009 school year, 
grades PK through 12. 

Note: Fiscal data are from 2007-2008 school year. 



American Institutes for Research * Council of the Great City Schools * Spring 2011 


97 



DISTRICT PROFILES 


Milwaukee: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 13 districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, San Diego, Jefferson County, Baltimore, Chicago, District 
of Columbia, and Los Angeles) in overall reading in 2009 after adjusting for relevant 
background characteristics. Milwaukee did not score higher than any other district in 
adjusted overall reading scores in 2009. 

• Average scale scores corresponded to the 22 nd percentile in both overall reading and the 
information subscale, and the 24 th percentile in literary subscale in 2009 on the national 
score distribution. The average student was below the first national quartile in all three 
measures. 

• 2009 was the first year Milwaukee participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 41 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Locate and recall information from text (three items), 
o Integrate and interpret information and ideas presented in text (two items). 
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Milwaukee: Grade 8 Reading 


• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than seven districts (Miami-Dade County, Boston, Austin, Atlanta, 
Charlotte, Houston, and Chicago) and higher than one district (Detroit) in overall reading 
in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 25 th percentile in both overall reading and 
information subscale, and to the 27 th percentile in literary subscale on the national score 
distribution. The average student was around the first national quartile in all three 
measures. 

• 2009 was the first year Milwaukee participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 53 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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DISTRICT PROFILES 


Milwaukee: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored higher than two districts (Fresno and Detroit) and lower than nine districts 
(Houston, Austin, Boston, Charlotte, New York City, Miami-Dade County, San Diego, 
Atlanta, and Baltimore) in overall mathematics in 2009 after adjusting for relevant 
background characteristics. 

• Average overall mathematics scale score corresponded to the 24 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
24 th (numbers) to 29 th (geometry). The average student was around the first national 
quartile in all subscales. 

• 2009 was the first year Milwaukee participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 42 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Add and subtract whole numbers, or fractions with like denominators, or decimals 
through hundredths. 

o Identify place value and actual value of digits in whole numbers, 
o Represent numbers using models such as base 10 representations, number lines and 
two-dimensional models. 

o Graph or interpret points with whole number or letter coordinates on grids or in the 
first quadrant of the coordinate plane, 
o Order/compare whole numbers, decimals, or fractions. 
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Milwaukee: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored higher than one district (Detroit) and lower than 13 districts (Austin, Boston, 
Houston, Charlotte, Miami-Dade County, New York City, San Diego, Chicago, 
Philadelphia, Atlanta, Cleveland, Baltimore, and Los Angeles) in overall mathematics in 
2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 20 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
19 th (algebra) to 24 th (numbers). The average student was below the first national quartile 
in all subscales. 

• 2009 was the first year Milwaukee participated in NAEP TUDA. Thus, there are no trend 
data for this district. 

• On average, students answered 35 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Use place value to model and describe integers and decimals. 

o Interpret relationships between symbolic linear expressions and graphs of lines by 
identifying and computing slope and intercepts, 
o Compare objects with respect to length, area, volume, angle measurement, weight, or 
mass. 

o Determine the probability of independent and dependent events, 
o Interpret probabilities within a given context. 



New York City 


New York City participated in grade 4 and grade 8 NAEP reading and mathematics assessments 
in both 2007 and 2009. It had the fourth lowest student-teacher ratio among the 18 TUDA 
districts. Seventy-five percent of total expenditures were instructional. 

Table 60. New York City’s Demographics, 2009 


Number of Schools 

1452 

Number of Students 

960,553 

Student/Teacher Ratio 

14.3 

Free and Reduced-Price Lunch 

67% 

Expenditures ($/student) 

Total 

17,923 

Instructional 

13,529 

Student and Staff Support 

306 

Administration 

1,226 

Operations, Food Service, and Other Support Staff 

2,861 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


New York City: Grade 4 Reading 

• Scored no differently from the national and higher than the LC averages in overall 
reading in 2009. 

• Scored higher than 12 districts (Atlanta, San Diego, Jefferson County, Baltimore, 
Chicago, District of Columbia, Los Angeles, Milwaukee, Fresno, Philadelphia, 
Cleveland, and Detroit) in overall reading in 2009 after adjusting for relevant background 
characteristics. No district scored higher than New York City in adjusted overall reading 
scores in 2009. 

• Average scale scores corresponded to the 42 nd percentile in both overall reading and the 
information subscales, and the 44 th percentile in the literary subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed significant gain in both overall reading and information subscale from 2007 to 
2009. 


Table 61. New York City's Changes in Grade 4 Reading Overall and Subscale Scores, 
2007-2009 


National Public 


Large City 


New York City 

Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.10 

t 

Information 

0.01 

o 

0.06 

t 

0.13 

t 

Literary 

-0.01 

o 

0.03 

O 

0.08 

O 


Note. 1 Significant increase, <-> Change not significant, J Significant decrease 


• On average, students answered 53 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Locate and recall information from text (three items), 
o Integrate and interpret information and ideas presented in text (two items). 
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New York City: Grade 8 Reading 


• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored higher than three districts (District of Columbia, Fresno, and Detroit) and lower 
than three districts (Miami-Dade County, Boston, and Austin) in overall reading in 2009 
after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 36 th percentile on the national score distribution 
for overall reading, and on the literary and information subscales. The average student 
was below the national median in all three measures. 

• Displayed no significant change in overall reading or reading subscales from 2007 to 
2009. 


Table 62. New York City's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


New York City 

Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.09 

O 

Information 

0.04 

t 

0.04 

O 

0.04 

O 

Literary 

0.02 

t 

0.05 

t 

0.09 

O 


Note, f Significant increase, <— > Change not significant, [ Significant decrease 


• On average, students answered 60 percent of items correctly. The five differentially most 
difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text. 

o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 
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DISTRICT PROFILES 


New York City: Grade 4 Mathematics 

• Scored no differently from the national and higher than the LC averages in overall 
mathematics in 2009. 

• Scored higher than 13 districts (all except Austin, Boston, Charlotte, and Houston) in 
overall mathematics in 2009 after adjusting for relevant background characteristics. No 
district scored higher than New York City in adjusted overall mathematics scores in 
2009. 

• Average overall mathematics scale score corresponded to the 46 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
42 nd (data) to 48 th (numbers). The average student was below the national median in data 
and algebra, and around the national median in the other three subscales. 

• Displayed no significant change in overall mathematics or in the mathematics subscales 
from 2007 to 2009. 

Table 63. New York City's Changes in Grade 4 Mathematics Overall and Subscales Scores, 
2007-2009 



National Public 

Large City 


New York City 


Effect Size 

Significance 

Effect Size Significance 

Effect Size 

Significance 

Composite 

0.00 

O 

0.05 

t 

0.05 

O 

Numbers 

0.01 

O 

0.05 

t 

0.05 

O 

Measurement 

-0.01 

o 

0.07 

t 

0.06 

o 

Geometry 

0.03 

t 

0.07 

t 

0.10 

o 

Data 

-0.03 

t 

0.01 

O 

0.03 

o 

Algebra 

0.00 

o 

0.03 

O 

0.00 

o 


Note, f Significant increase, <-> Change not significant, J, Significant decrease 


• On average, students answered 52 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Read or interpret a single set of data. 

o Determine a simple probability from a context that includes a picture, 
o Identify the images resulting from flips, slides, or turns. 

o Determine appropriate size of unit of measurement in problem situation involving 
such attributes as length, time, capacity, or weight, 
o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments. 
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New York City: Grade 8 Mathematics 


• Scored lower than the national and no differently from the LC average in overall 
mathematics in 2009. 

• Scored higher than eight districts (Cleveland, Baltimore, Los Angeles, Jefferson County, 
District of Columbia, Milwaukee, Fresno, and Detroit) and lower than four districts 
(Austin, Boston, Houston, and Charlotte) in overall mathematics in 2009 after adjusting 
for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 40 th percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
36 th (data) to 42 nd (algebra). The average student was below the national median in all 
subscales. 

• Displayed no significant change in overall mathematics or in the mathematics subscales 
from 2007 to 2009. 


Table 64. New York City's Changes in Grade 8 Mathematics Overall and Subscale Scores, 
2007-2009 



National Public 

Large City 


New York City 


Effect Size 

Significance 

Effect Size Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.08 

O 

Numbers 

0.02 

t 

0.09 

t 

0.11 

O 

Measurement 

0.04 

t 

0.08 

t 

0.12 

O 

Geometry 

0.05 

t 

0.07 

t 

0.07 

O 

Data 

0.00 

O 

0.02 

O 

0.01 

O 

Algebra 

0.06 

t 

0.05 

O 

0.08 

O 


Note. | Significant increase, <-> Change not significant, J Significant decrease 


• On average, students answered 46 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Read or interpret data, including interpolating or extrapolating from data, 
o Select or use an appropriate type of unit for the attribute being measured such as 
length, area, angle, time, or volume, 
o Use place value to model and describe integers and decimals, 
o Write algebraic expressions, equations, or inequalities to represent a situation, 
o Identify, define, or describe geometric shapes in the plane and in three-dimensional 
space given a visual representation. 
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Philadelphia 


Philadelphia participated in grade 4 and grade 8 NAEP reading and mathematics assessments in 
2009. It had the eighth highest student-teacher ratio among the 18 TUDA districts. Fifty-four 
percent of total expenditures were instructional. 

Table 65. Philadelphia's Demographics, 2009 


Number of Schools 

275 

Number of Students 

159,867 

Student/Teacher Ratio 

15.6 

Free and Reduced-Price Lunch 

73% 

Expenditures ($/student) 

Total 

9,399 

Instructional 

5,051 

Student and Staff Support 

782 

Administration 

1,117 

Operations, Food Service and Other Support Staff 

2,449 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


Philadelphia: Grade 4 Reading 

• Scored lower than the national and LC averages in overall reading in 2009. 

• Scored lower than 13 districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, Houston, Atlanta, San Diego, Jefferson County, Baltimore, Chicago, District 
of Columbia, and Los Angeles) in overall reading in 2009 after adjusting for relevant 
background characteristics. Philadelphia did not score higher than any other district in 
adjusted overall reading scores in 2009. 

• Average scale scores in overall reading, and in the literary and information subscales 
corresponded to the 22 nd , 24 th , and 21 st percentiles, respectively, on the national score 
distribution. The average student was below the first national quartile in all three 
measures. 

• 2009 was the first year Philadelphia participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 42 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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Philadelphia: Grade 8 Reading 


• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored lower than three districts (Miami-Dade County, Boston, and Austin) and higher 
than two districts (District of Columbia and Detroit) in overall reading in 2009 after 
adjusting for relevant background characteristics. 

• Average scale scores in overall reading, and in the literary and information subscales 
corresponded to the 30 th , 32 nd , and 29 th percentiles, respectively, on the national score 
distribution. The average student was below the first national quartile in all three 
measures. 

• 2009 was the first year Philadelphia participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 56 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (four items), 
o Critique and evaluate information and ideas in text and the ways in which authors 
present text. 


DISTRICT PROFILES 


Philadelphia: Grade 4 Mathematics 

• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than nine districts (Houston, Austin, Boston, Charlotte, New York City, 
Miami-Dade County, San Diego, Atlanta, and Baltimore) and higher than three districts 
(Cleveland, Fresno, and Detroit) in overall mathematics in 2009 after adjusting for 
relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 26 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
24 th (data) to 29 th (numbers). The average student was around the first national quartile in 
all subscales except numbers. 

• 2009 was the first year Philadelphia participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 43 percents of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o For a given set of data, complete a graph (two items). 

o Identify the images resulting from flips (reflections), slides (translations), or turns 
(rotations). 

o Select or use appropriate measurement instruments such as ruler, meter stick, clock, 
thermometer, or other scaled instruments, 
o Order/compare whole numbers, decimals, or fractions. 
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Philadelphia: Grade 8 Mathematics 


• Scored lower than the national and LC averages in overall mathematics in 2009. 

• Scored lower than five districts (Austin, Boston, Houston, Charlotte, and Miami-Dade 
County) and higher than six districts (Los Angeles, Jefferson County, District of 
Columbia, Milwaukee, Fresno, and Detroit) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 31 st percentile on the 
national score distribution. The percentiles for the average subscale scores ranged from 
30 th (numbers and algebra) to 34 th (data). The average student was below national median 
in all subscales. 

• 2009 was the first year Philadelphia participated in NAEP TUDA. Thus, there are no 
trend data for this district. 

• On average, students answered 42 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Explain or justify a mathematical concept or relationship, 
o Perform computations with rational numbers, 
o Use place value to model and describe integers and decimals. 

o Graph or interpret points represented by ordered pairs of numbers on a rectangular 
coordinate system. 

o Solve problems involving coordinate pairs on the rectangular coordinate system. 
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San Diego 


San Diego participated in grade 4 and grade 8 NAEP reading and mathematics assessments in 
both 2007 and 2009. It had the fourth highest student-teacher ratio among the 18 TUDA districts. 
Fifty- six percent of total expenditures were instructional. 

Table 66. San Diego's Demographics, 2009 


Number of Schools 

223 

Number of Students 

132,256 

Student/Teacher Ratio 

19.3 

Free and Reduced-Price Lunch 

63% 

Expenditures ($/student) 

Total 

10,305 

Instructional 

5,767 

Student and Staff Support 

1,496 

Administration 

1,225 

Operations, Food Service and Other Support Staff 

1,817 


Source: Common Core of Data public school district data for the 2008-2009 school year, grades PK through 12. 
Note: Fiscal data are from 2007-2008 school year. 
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DISTRICT PROFILES 


San Diego: Grade 4 Reading 

• Scored lower than the national and no differently from the LC average in overall reading 
in 2009. 

• Scored lower than six districts (Boston, Miami-Dade County, New York City, Austin, 
Charlotte, and Houston) and higher than eight districts (Chicago, District of Columbia, 
Los Angeles, Milwaukee, Fresno, Philadelphia, Cleveland, and Detroit) in overall reading 
in 2009 after adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 38 th percentile in both overall reading and the 
information subscale, and the 39 th percentile in literary subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed no significant change in overall reading or the reading subscales from 2007 to 
2009. 


Table 67. San Diego's Changes in Grade 4 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


San Diego 

Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.00 

o 

0.04 

O 

0.07 

O 

Information 

0.01 

o 

0.06 

t 

0.09 

O 

Literary 

-0.01 

o 

0.03 

O 

0.05 

O 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 53 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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San Diego: Grade 8 Reading 


• Scored lower than the national and no differently from the LC averages in overall reading 
in 2009. 

• Scored lower than three districts (Miami-Dade County, Boston, and Austin) and higher 
than two districts (District of Columbia, and Detroit) in overall reading in 2009 after 
adjusting for relevant background characteristics. 

• Average scale scores corresponded to the 38 th percentile in overall reading and the 
literary subscale, and the 39 th percentile in the information subscale on the national score 
distribution. The average student was below the national median in all three measures. 

• Displayed no significant change in overall reading or in the reading subscales from 2007 
to 2009. 


Table 68. San Diego's Changes in Grade 8 Reading Overall and Subscale Scores, 2007-2009 


National Public 


Large City 


San Diego 


Effect Size Significance 

Effect Size 

Significance 

Effect Size 

Significance 

Composite 

0.04 

t 

0.07 

t 

0.11 

O 

Information 

0.04 

t 

0.04 

O 

0.10 

O 

Literary 

0.02 

t 

0.05 

t 

0.09 

O 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 62 percent of items correctly. The top five differentially 
most difficult items measured the following cognitive targets: 

o Integrate and interpret information and ideas presented in text (three items), 
o Locate and recall information from text (two items). 
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DISTRICT PROFILES 


San Diego: Grade 4 Mathematics 

• Scored no differently from the national and higher than the LC averages in overall 
mathematics in 2009. 

• Scored lower than five districts (Houston, Austin, Boston, Charlotte, and New York City) 
and higher than nine districts (Philadelphia, Jefferson County, Chicago, District of 
Columbia, Los Angeles, Milwaukee, Cleveland, Fresno, and Detroit) in overall 
mathematics in 2009 after adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 44 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
41 st (data) to 51 st (geometry). The average student was below the national median in data, 
algebra, and measurement, around the national median in numbers, and just above the 
national median in geometry. 

• Displayed no significant change in overall mathematics or in the mathematics subscales 
from 2007 to 2009. 

Table 69. San Diego's Changes in Grade 4 Mathematics Overall and Subscales Scores, 2007- 
2009 



National Public 

Large City 


San Diego 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.00 

O 

0.05 

t 

0.07 

O 

Numbers 

0.01 

O 

0.05 

t 

0.10 

O 

Measurement 

-0.01 

O 

0.07 

t 

0.08 

O 

Geometry 

0.03 

t 

0.07 

t 

0.08 

O 

Data 

-0.03 

t 

0.01 

O 

0.07 

O 

Algebra 

0.00 

O 

0.03 

o 

-0.04 

O 


Note, f Significant increase, <-> Change not significant, ( Significant decrease 


• On average, students answered 53 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Verify a conclusion using algebraic properties, 
o Identify factors of whole numbers, 
o Multiply whole numbers. 

o Graph or interpret points with whole number or letter coordinates on grids or in the 
first quadrant of the coordinate plane. 

o Represent the probability of a given outcome using a picture or other graphic. 
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San Diego: Grade 8 Mathematics 


• Scored no differently from the national and higher than the LC averages in overall 
mathematics in 2009. 

• Scored lower than four districts (Austin, Boston, Houston, and Charlotte) and higher than 
eight districts (Cleveland, Baltimore, Los Angeles, Jefferson County, District of 
Columbia, Milwaukee, Fresno, and Detroit) in overall mathematics in 2009 after 
adjusting for relevant background characteristics. 

• Average overall mathematics scale score corresponded to the 47 th percentile on the 
national score distribution. The percentile for the average subscale scores ranged from 
39 th (data) to 53 ld (algebra). The average student was above the national median in 
algebra, around the national median in numbers and geometry, and below the national 
median in the other two subscales. 

• Displayed significant gains in overall mathematics and in the numbers and algebra 
subscales from 2007 to 2009. 

Table 70. San Diego's Changes in Grade 8 Mathematics Overall and Subscale Scores, 2007- 
2009 



National Public 

Large City 


San Diego 



Effect Size 

Significance 

Effect Size Significance 

Effect Size Significance 

Composite 

0.04 

t 

0.07 

t 

0.20 

t 

Numbers 

0.02 

t 

0.09 

t 

0.24 

t 

Measurement 

0.04 

t 

0.08 

t 

0.13 

O 

Geometry 

0.05 

t 

0.07 

t 

0.26 

O 

Data 

0.00 

O 

0.02 

O 

0.09 

O 

Algebra 

0.06 

t 

0.05 

o 

0.22 

t 


Note. ( Significant increase, <-> Change not significant, [ Significant decrease 


• On average, students answered 49 percent of items correctly. The top five differentially 
most difficult items measured the following objectives: 

o Recognize or informally describe the effect of a transformation on two-dimensional 
geometric shapes (two items). 

o Estimate the size of an object with respect to a given measurement attribute, 
o Use proportional reasoning to model and solve problems (including rates and 
scaling). 

o Determine the sample space for a given situation. 
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DISCUSSION 


Discussion 

In this report, we examined the performance of 18 districts that participated in the 2009 NAEP 
grade 4 and 8 reading and mathematics assessments. Eleven of these districts had also 
participated in 2007 assessments. We analyzed the performance of all 18 districts in 2009 and 
also examined the changes in performance for the 11 districts from 2007 to 2009. 

It is evident that the academic performance of public school students in many of the urban 
districts we examined this report is nowhere near what we would like it to be. However, the story 
is not uniform across all districts. There are districts that perform similar to, and, in some cases, 
even higher than the national average. Charlotte, Boston, and Austin are three examples. We also 
see districts that are performing below the large city and the national averages, yet are making 
significant progress. An example is the District of Columbia where significant gains were 
observed in reading and mathematics at both grades. 

On the other hand, some districts have a longer path to travel in order to achieve their targets. For 
example, among the 11 districts that participated in the 2007 and 2009 NAEP assessments, 
Cleveland and Chicago were the only two districts that performed lower than the national and the 
large city averages and showed no gains from 2007 to 2009. 

Policy makers, researchers and practitioners will be carefully watching the future performance of 
the nine TUDA districts that participated in NAEP for the first time in 2009. Despite their 
starting points, will Baltimore City, Detroit, Fresno, Milwaukee, and Philadelphia show progress 
in future assessments? Will Jefferson County perform as well in mathematics as it does in 
reading? 

Knowing where one is and knowing where one is headed are the first steps in making better 
decisions about reaching future targets. Like several other studies that use NAEP data, this study 
illustrates the depth and wealth of information available about academic performance of public 
school students in urban districts in the United States. Policy makers and practitioners can use 
this information. The variation in the demographic profiles of the 18 urban districts examined in 
this report makes the case that there is much these districts can learn from each other. 
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APPENDIX A 


Appendix A. Adjusted Mean Scores 
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Variables Used in Regression Analyses to Calculate "Adjusted" Scores 

• Race/ethnicity 

In the NAEP files, student race/ethnicity information is obtained from school records and 
classified under six categories: White, Black , Hispanic, Asian/Pacific Islander, American 
Indian/Alaska Native, or unclassifiable. When school-reported information was missing, 
student-reported data from the Student Background Questionnaire were used to establish 
race/ethnicity. We categorized as unclassifiable the students whose race-ethnicity based on 
school-records was unclassifiable or missing and (1) who self-reported their race as 
multicultural but not Hispanic or (2) who did not self-report race information. 

• Special education status 

Student has an Individualized Educational Program (1EP), for reasons other than being gifted 
or talented; or a student with a Section 504 Plan. 

• English language learner status 

Student is currently classified as an English language learner and is receiving services. 

• Free- or reduced-price lunch eligibility 

Eligibility for the National School Lunch Program is determined by a student’s family income 
in relation to the federally established poverty level. Based on available school records, 
students were classified as either currently eligible for free/reduced-price lunch or currently 
not eligible. If the school record indicated the information was not available, the student was 
classified as not eligible. 

• Parental Education 

Highest level of education attained by either parent: did not complete high school, graduate 
high school, some education after high school, and graduated college. This indicator is only 
available for grade 8 students. 

• Literacy Materials 

The presence of literacy materials in the home is associated with both socioeconomic status 
and student achievement. The measure reported here is based on questions in both the grade 4 
and grade 8 Student Background Questionnaires that ask about the availability of computer, 
newspapers, magazines, and more than 25 books in the home. A summary score has been 
created to indicate how many of these four types of literacy materials are present. 13 

Information on race/ethnicity, free-lunch, ELL and SD status come from the school and are 
available for all students. However, data on background characteristics for students that do not 
participate in NAEP are not available: excluded students do not fill the Background 
Questionnaire. Therefore, data on literacy materials and parent education are only available for 
the included population. Therefore, the calculation of adjusted scores controlling for background 
characteristics was conducted on the reported sample only. 


13 This summary score has been used for reporting NAEP background variables for a number of years and has been 
shown to be associated with students’ achievement scores (for example, NAEP 1996 Mathematics Cross-State Data 
Compendium) 
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Appendix A (continued) 

Estimating adjusted mean scores 

The method used in calculating the adjusted district means is discussed below. 

Let y be plausible value v of student j in district i, and 

X Jk be the demographic characteristic k of student j in district i. 

Assume the mean plausible value student j in district i, y tj ., can be expressed as a function of an 
overall mean achievement ju, a differential effect a j associated with district i, and differential 
effects P k associate with characteristic k of student j in district i: 

y v . = » + a i + YjPk x ijk +e iji [!] 

where n is the overall mean, 

a, is the district i effect, and 

p k is the effect of the demographic characteristic k of student j in district i. 

Letting the subscript • indicate average, then the average scale score in district i is expressed as 

A.. =/J + a t + Y, Pk x i.k + e \ > [2] 

k 

Subtracting [2] from [1] we can estimate the regression in [3] 

Ay = y,j. - y,.. = Z Pkl x uk - x t.k\ + e 'ij [3] 

and obtain estimates of p k directly, without any contamination from the a because a , has been 
subtracted out before the regression. 

With the estimates P t , we compute the average effect of the demographic characteristics of 
student j in district i. 

y IJ . = JlPdX m - x .. k ] [4] 

where X %tk is the overall mean of X t%k . 

The adjusted score, y' ijv is estimated by subtracting y jm from each y ijv : 

y , 9 v=y 9 v-yt. [5] 

The adjusted score, y\„ is the critical statistic for the analysis. It is an estimator for // + a t and 
we can estimate its standard error by the usual NAEP procedures. Note that // + a t is the overall 
mean plus the effect of district i. It is what the mean of district i would be if the mean of all 
demographics in district i were the same as the overall mean of demographics. 
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APPENDIX B 


Appendix B. Average Scores by Subscale and District: 

2009 
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Table Bl. Average Grade 4 Reading Scores, by Subscale and Jurisdiction: 2009 


State/jurisdiction 

Composite 

Information 

Literary 

National Public 

220 (0.3) 

218 (0.3) 

221 (0.3) 

Large City 

210(0.7) 

207 (0.8) 

212(0.7) 

Atlanta 

209** (1.5) 

206** ( 2 . 2 ) 

212 ** ( 2 . 8 ) 

Austin 

220 * ( 1 . 8 ) 

219* (2.8) 

222 * ( 2 ) 

Baltimore City 

202 *’**(1.7) 

199*’** (2.5) 

204*'** (1.8) 

Boston 

215*’** (1.2) 

211 ** ( 1 . 6 ) 

219* (1) 

Charlotte 

225* ** (1.6) 

223 *. ** ( 2 ) 

226*’** ( 1 . 8 ) 

Chicago 

202*’** (1.5) 

199* ** (i. 6 ) 

204*’** (1.7) 

Cleveland 

194* ** ( 2 ) 

192* ** (1.6) 

195*’** (2.8) 

Detroit 

187* ** (1.9) 

183*’** (3) 

190*’** (1.8) 

District of Columbia 

203*’** (1.2) 

199* ** (i.i) 

207*’** (1.6) 

Fresno 

197*.** (i 7 ) 

192* ** (2.6) 

202*’** (1.9) 

Houston 

211** (1.7) 

208** ( 2 ) 

214** (2) 

Jefferson County 

219* (1.8) 

219* (2) 

219* (1.9) 

Los Angeles 

197*’** (1.1) 

194*’** (1.1) 

200*’** (1.4) 

Miami-Dade County 

221 * ( 1 . 2 ) 

219* (1.6) 

223* (1.3) 

Milwaukee 

196* ** (2) 

192* ** ( 2 . 5 ) 

199 * ** ( 2 . 2 ) 

New York City 

217* (1.4) 

214* ** (1.3) 

219* (1.7) 

Philadelphia 

195* ** (i. 8 ) 

191* ** (2.6) 

199*’** (1.7) 

San Diego 

213** (2.1) 

210** (2.3) 

215 (3.4) 

*Significantly different (p < .05) from large city 


**Significantly different (p < .05) from national public 
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Table B2. Average Grade 8 Reading Scores, by Subscale and Jurisdiction: 2009 


State/jurisdiction 

Composite 

Information 

Literary 

National Public 

262 (0.3) 

264 (0.3) 

261 (0.3) 

Large City 

252 (0.5) 

254 (0.6) 

251 (0.6) 

Atlanta 

250** (1.5) 

253** (1.2) 

246*’** (2.2) 

Austin 

261* ( 2 ) 

263* (2.8) 

259* (2.9) 

Baltimore City 

245*'** (1.7) 

247*’** (1.8) 

242*’** (2.4) 

Boston 

257*'** (1.5) 

258*'** (2) 

257*’** (2) 

Charlotte 

259*'** (1) 

261*’** ( 1 . 2 ) 

258* (1.3) 

Chicago 

249** (1.6) 

250** (1.7) 

248** (1.8) 

Cleveland 

242* ** ( 1 . 6 ) 

244 *’** (2.9) 

240*’** (1.5) 

Detroit 

232*'** (2.4) 

232*'** (2.4) 

233*’** (4) 

District of Columbia 

240* ** (1.5) 

242*’** (2.1) 

238*’** (1.2) 

Fresno 

240*'** (2.4) 

240*’** (2.9) 

239*’** (2.4) 

Houston 

252** (1.2) 

254** (1.8) 

250** (1.6) 

Jefferson County 

259***(i) 

262* (1.3) 

254*’** (1.2) 

Los Angeles 

244*'** (1.1) 

245*’** (1.6) 

242*’** (1.1) 

Miami-Dade County 

261* (1.4) 

262* (1.5) 

260* ( 1 . 6 ) 

Milwaukee 

241*'** (2) 

242*’** (2.3) 

241*’** (2.4) 

New York City 

252** (1.4) 

255** (1.5) 

250** (1.6) 

Philadelphia 

247** (2.5) 

248** (3) 

246** (2.1) 

San Diego 

254** (2.8) 

257 (3.8) 

252** (2.3) 

*Significantly different (p < .05) from large city 

**Significantly different (p < .05) from national public 
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Table B3. Average Grade 4 Mathematics Scores, by Subscale and Jurisdiction: 2009 
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Table B4. Average Grade 8 Mathematics Scores, by Subscale and Jurisdiction: 2009 


APPENDIX B 



ADDENDUM TO PIECES OF THE PUZZLE: RECENT PERFORMANCE TRENDS IN URBAN DISTRICTS 




APPENDIX C 


Appendix C. Average Scores Adjusted for Relevant 
Background Variables, by District: 2009 
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Table Cl. Average Scale Scores of Public School Students, Adiusted for Relevant Background Variables, in 2009 Grade 4 NAEP Reading Assessment, by 
District 
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Table C2. Average Scale Scores of Public School Students, Adjusted for Relevant Background Variables, in 2009 Grade 8 NAEP Reading Assessment, by 
District 
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Table C3. Average Scale Scores of Public School Students, Adiusted for Relevant Background Variables, in 2009 Grade 4 NAEP Mathematics Assessment, 
by District 
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Table C4. Average Scale Scores of Public School Students, Adiusted for Relevant Background Variables, in 2009 Grade 8 NAEP Mathematics Assessment, 
by District 
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APPENDIX D 


Appendix D. Average Expected Scores Based on 
Relevant Background Variables and District Effects, by 

District: 2009 
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Table Dl. Average Expected Scale Scores of Public School Students, Based on Relevant 
Background Variables, in 2009 Grade 4 NAEP Reading, by District 


City/jurisdiction 

Mean 

Expected mean 

District effect 

Atlanta 

209.1 

208.2 

0.9 

Austin 

220.4 

213.9 

6.5* 

Baltimore City 

202.0 

206.3 

-4.3* 

Boston 

215.0 

206.4 

8.6* 

Charlotte 

224.4 

218.2 

6.2* 

Chicago 

202.2 

206.8 

-4.6* 

Cleveland 

193.6 

206.1 

-12.4* 

Detroit 

187.2 

203.1 

-15.9* 

District of Columbia 

203.5 

209.4 

-5.9* 

Fresno 

197.3 

208.3 

-11.0* 

Houston 

211.4 

206.7 

4.7* 

Jefferson County 

219.4 

220.4 

-1.0 

Los Angeles 

197.4 

203.7 

-6.3* 

Miami-Dade County 

221.2 

213.0 

8.1* 

Milwaukee 

195.8 

206.6 

-10.8* 

New York City 

216.8 

209.6 

7.2* 

Philadelphia 

195.0 

207.1 

-12.1* 

San Diego 

212.8 

212.6 

0.2 


Note. District effect is the difference between district mean and expected district mean. 
* District effect is significantly different from zero 
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APPENDIX D 


Table D2. Average Expected Scale Scores of Public School Students, Based on Relevant 
Background Variables, in 2009 Grade 8 NAEP Reading, by District 


City/jurisdiction 

Mean 

Expected mean 

District effect 

Atlanta 

249.7 

246.9 

2.8 

Austin 

261.1 

254.9 

6.1* 

Baltimore City 

244.6 

246.5 

-1.9 

Boston 

257.3 

250.7 

6.6* 

Charlotte 

259.3 

256.8 

2.5* 

Chicago 

249.1 

247.7 

1.5 

Cleveland 

242.3 

244.4 

-2.1 

Detroit 

232.2 

242.8 

-10.7* 

District of Columbia 

240.3 

247.6 

-7.3* 

Fresno 

239.6 

247.8 

-8.1* 

Houston 

251.9 

249.6 

2.2* 

Jefferson County 

258.5 

261.4 

-2.9* 

Los Angeles 

243.8 

245.7 

-1.9* 

Miami-Dade County 

260.6 

253.1 

7.5* 

Milwaukee 

241.4 

245.9 

-4.6* 

New York City 

252.4 

252.8 

-0.4 

Philadelphia 

247.0 

248.3 

-1.3 

San Diego 

254.4 

255.6 

-1.2 


Note. District effect is the difference between district mean and expected district mean. 
* District effect is significantly different from zero 
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Table D3. Average Expected Scale Scores of Public School Students, Based on Relevant 
Background Variables, in 2009 Grade 4 NAEP Mathematics, by District 



Note. District effect is the difference between district mean and expected district mean. 
* District effect is significantly different from zero 
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APPENDIX D 


Table D4. Average Expected Scale Scores of Public School Students, Based on Relevant 
Background Variables, in 2009 Grade 8 NAEP Mathematics, by District 


City/jurisdiction 

Mean 

Expected mean 

District effect 

Atlanta 

259.4 

260.5 

-1.1 

Austin 

287.2 

272.8 

14.4* 

Baltimore City 

257.1 

260.1 

-3.0 

Boston 

279.4 

267.3 

12.1* 

Charlotte 

282.4 

274.0 

8.4* 

Chicago 

263.6 

263.6 

0.0 

Cleveland 

255.7 

258.3 

-2.6* 

Detroit 

238.1 

256.4 

-18.3* 

District of Columbia 

251.1 

259.4 

-8.4* 

Fresno 

258.3 

268.3 

-9.9* 

Houston 

276.9 

265.8 

11.1* 

Jefferson County 

271.1 

278.2 

-7.1* 

Los Angeles 

258.4 

264.5 

-6.1* 

Miami-Dade County 

272.7 

269.1 

3.6* 

Milwaukee 

251.2 

260.8 

-9.5* 

New York City 

272.8 

270.1 

2.7* 

Philadelphia 

264.5 

265.0 

-0.5 

San Diego 

280.1 

278.1 

2.0 


Note. District effect is the difference between district mean and expected district mean. 
* District effect is significantly different from zero 
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APPENDIX E 


Appendix E. Average Scores Expressed in Percentiles, by 

Subscale and District: 2009 
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Table El. Average Subscale and Composite Scale Scores Expressed in Percentiles on the 
National Public Score Distribution for 2009 Grade 4 NAEP Reading Assessment, by District 


City/jurisdiction 

Composite 

Literary 

Information 

Atlanta 

34 

36 

33 

Austin 

46 

46 

47 

Baltimore City 

28 

28 

27 

Boston 

40 

43 

38 

Charlotte 

51 

51 

51 

Chicago 

28 

29 

28 

Cleveland 

21 

21 

22 

Detroit 

16 

18 

16 

District of Columbia 

29 

31 

28 

Fresno 

23 

27 

21 

Houston 

36 

38 

36 

Jefferson County 

45 

44 

47 

Los Angeles 

24 

25 

23 

Miami-Dade County 

47 

48 

47 

Milwaukee 

22 

24 

22 

New York City 

42 

44 

42 

Philadelphia 

22 

24 

21 

San Diego 

38 

39 

38 



American Institutes for Research * Council of the Great City Schools • Spring 2011 


147 



APPENDIX E 


Table E2. Average Subscale and Composite Scale Scores Expressed in Percentiles on the 
National Public Score Distribution for 2009 Grade 8 NAEP Reading Assessment, by District 


City/jurisdiction 

Composite 

Literary 

Information 

Atlanta 

33 

32 

35 

Austin 

46 

45 

46 

Baltimore City 

28 

28 

29 

Boston 

41 

42 

41 

Charlotte 

44 

44 

43 

Chicago 

32 

34 

32 

Cleveland 

26 

27 

26 

Detroit 

18 

21 

17 

District of Columbia 

24 

25 

25 

Fresno 

23 

25 

23 

Houston 

35 

36 

35 

Jefferson County 

42 

40 

45 

Los Angeles 

27 

28 

27 

Miami-Dade County 

45 

46 

44 

Milwaukee 

25 

27 

25 

New York City 

36 

36 

36 

Philadelphia 

30 

32 

29 

San Diego 

38 

38 

39 
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Table E3. Average Subscale and Composite Scale Scores Expressed in Percentiles on the 
National Public Score Distribution for 2009 Grade 4 NAEP Mathematics Assessment, by 
District 


City/jurisdiction 

Composite 

Number 

Measure 

Geometry 

Data 

Algebra 

Atlanta 

30 

32 

30 

33 

27 

33 

Austin 

50 

51 

50 

48 

46 

48 

Baltimore 

27 

29 

22 

30 

34 

28 

Boston 

44 

46 

44 

50 

38 

38 

Charlotte 

56 

57 

49 

56 

60 

60 

Chicago 

27 

26 

30 

30 

28 

25 

Cleveland 

18 

18 

20 

24 

20 

21 

Detroit 

9 

9 

10 

13 

11 

12 

District of Columbia 

24 

25 

24 

25 

28 

29 

Fresno 

23 

29 

19 

24 

22 

24 

Houston 

44 

44 

47 

38 

40 

44 

Jefferson 

40 

36 

42 

48 

43 

39 

Los Angeles 

26 

30 

23 

25 

25 

29 

Miami-Dade County 

44 

41 

44 

48 

51 

46 

Milwaukee 

24 

24 

26 

29 

27 

26 

New York City 

46 

48 

46 

45 

42 

44 

Philadelphia 

26 

29 

28 

25 

24 

26 

San Diego 

44 

45 

43 

51 

41 

42 
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Table E4. Average Subscale and Composite Scale Scores Expressed in Percentiles on the 
National Public Score Distribution for 2009 Grade 8 NAEP Mathematics Assessment, by 
District 


City/jurisdiction 

Composite 

Number 

Measure 

Geometry 

Data 

Algebra 

Atlanta 

26 

26 

23 

29 

28 

29 

Austin 

55 

51 

59 

61 

53 

50 

Baltimore 

24 

26 

26 

25 

29 

23 

Boston 

47 

46 

49 

45 

49 

45 

Charlotte 

50 

45 

49 

55 

50 

48 

Chicago 

30 

30 

33 

31 

31 

30 

Cleveland 

23 

26 

23 

26 

23 

24 

Detroit 

12 

14 

12 

12 

12 

14 

District of Columbia 

20 

23 

19 

19 

24 

20 

Fresno 

25 

31 

25 

24 

23 

27 

Houston 

43 

45 

47 

45 

43 

40 

Jefferson 

38 

36 

37 

39 

38 

39 

Los Angeles 

25 

30 

23 

25 

24 

28 

Miami-Dade County 

40 

36 

41 

40 

40 

40 

Milwaukee 

20 

24 

22 

21 

21 

19 

New York City 

40 

38 

41 

40 

36 

42 

Philadelphia 

31 

30 

33 

33 

34 

30 

San Diego 

47 

48 

43 

48 

39 

53 
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APPENDIX F 


Appendix F. Average Percentage Correct and Omission 

Rates by District: 2009 
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Tables FI. Average Percentage Correct and Omission-Rates in 2009 Grade 4 NAEP Reading Assessment by Subscale and Item Type 





American Institutes for Research * Council of the Great City Schools • Spring 2011 


153 



Tables F2. Average Percentage Correct and Omission-Rates In 2009 Grade 8 NAEP Reading Assessment by Subscale and Item Type 


APPENDIX F 
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MC= Multiple-choice, SCR= Short constructed response, ECR = Extended constructed response 




Tables F3.A. Average Percentage Correct Rates in 2009 Grade 4 NAEP Mathematics Assessment by Subscale, Item Type and Mathematical 
Complexity 




Note. MC= Multiple-choice, SCR= Short constructed response, ECR = Extended constructed response. 



Tables F3.B. Average Percentage Omission-Rates in 2009 Grade 4 NAEP Mathematics Assessment by Subscale, Item Type and Mathematical 
Complexity 


APPENDIX F 



MC= Multiple-choice, SCR= Short constructed response, ECR = Extended constructed response. 




Tables F4.A. Average Percentage Correct Rates in 2009 Grade 8 NAEP Mathematics Assessment by Subscale, Item Type and Mathematical 
Complexity 





Tables F4.B. Average Percentage Omission-Rates in 2009 Grade 8 NAEP Mathematics Assessment by Subscale, Item Type and Mathematical 
Complexity 


APPENDIX F 
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Note. MC= Multiple-choice, SCR= Short constructed response, ECR = Extended constructed response. 




APPENDIX G 


Appendix G. Characteristics of Differentially Difficult 

Items by District: 2009 
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Table Gl. Characteristics of the Top Five Differentially Most Difficult Items in 2009 Grade 4 
NAEP Reading Assessment, by District 







Percent Correct 

District/ 

jurisdiction 

Item 

Type 

Subscale 

Objective 

National 

Public 

LC 

District 

Atlanta 

1 

SCR 

Informational 

Integrate/Interpret 

41 

32 

24 

2 

MC 

Informational 

Integrate/Interpret 

74 

64 

58 

3 

MC 

Literary 

Integrate/Interpret 

67 

60 

51 

4 

MC 

Informational 

Integrate/Interpret 

78 

70 

63 

5 

MC 

Informational 

Locate/Recall 

76 

66 

56 

Austin 

1 

MC 

Literary 

Critique/Evaluate 

39 

39 

29 

2 

SCR 

Literary 

Critique/Evaluate 

55 

48 

49 

3 

MC 

Informational 

Integrate/Interpret 

74 

64 

62 

4 

MC 

Informational 

Integrate/Interpret 

73 

64 

62 

5 

MC 

Literary 

Integrate/Interpret 

76 

73 

66 

Baltimore 

1 

SCR 

Informational 

Critique/Evaluate 

45 

38 

23 

2 

MC 

Literary 

Integrate/Interpret 

57 

50 

32 

3 

MC 

Informational 

Integrate/Interpret 

78 

70 

58 

4 

MC 

Literary 

Locate/Recall 

62 

54 

38 

5 

MC 

Literary 

Locate/Recall 

50 

43 

27 

Boston 

1 

MC 

Informational 

Integrate/Interpret 

64 

60 

48 

2 

MC 

Informational 

Integrate/Interpret 

73 

66 

61 

3 

MC 

Informational 

Integrate/Interpret 

78 

70 

67 

4 

MC 

Literary 

Locate/Recall 

50 

43 

35 

5 

MC 

Informational 

Locate/Recall 

60 

54 

49 

Charlotte 

1 

SCR 

Literary 

Critique/Evaluate 

77 

74 

76 

2 

MC 

Informational 

Integrate/Interpret 

73 

67 

68 

3 

MC 

Informational 

Integrate/Interpret 

44 

39 

39 

4 

MC 

Literary 

Integrate/Interpret 

61 

56 

58 

5 

MC 

Literary 

Integrate/Interpret 

49 

44 

45 

Chicago 

1 

MC 

Literary 

Critique/Evaluate 

61 

53 

44 

2 

SCR 

Literary 

Integrate/Interpret 

49 

43 

30 

3 

MC 

Literary 

Integrate/Interpret 

64 

52 

47 

4 

MC 

Literary 

Locate/Recall 

59 

50 

42 
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5 

MC 

Informational 

Locate/Recall 

43 

35 

27 

Cleveland 

1 

MC 

Literary 

Integrate/Interpret 

63 

55 

38 

2 

MC 

Literary 

Integrate/Interpret 

71 

67 

48 

3 

MC 

Literary 

Locate/Recall 

62 

54 

32 

4 

MC 

Informational 

Locate/Recall 

76 

66 

52 

5 

MC 

Literary 

Locate/Recall 

50 

43 

26 

Detroit 

1 

MC 

Literary 

Critique/Evaluate 

58 

54 

25 

2 

MC 

Literary 

Critique/Evaluate 

61 

53 

29 

3 

MC 

Informational 

Integrate/Interpret 

78 

70 

41 

4 

SCR 

Informational 

Integrate/Interpret 

65 

58 

28 

5 

MC 

Informational 

Locate/Recall 

76 

66 

41 

District of 
Columbia 

1 

SCR 

Informational 

Integrate/Interpret 

65 

58 

46 

2 

MC 

Informational 

Locate/Recall 

43 

35 

20 

3 

MC 

Informational 

Locate/Recall 

66 

60 

44 

4 

MC 

Informational 

Locate/Recall 

50 

44 

31 

5 

MC 

Literary 

Locate/Recall 

50 

43 

33 






Percent Correct 

District/ 

jurisdiction 

Item 

Type 

Subscale 

Objective 

National 

Public 

LC 

District 

Fresno 

1 

MC 

Informational 

Integrate/Interpret 

78 

70 

53 

2 

SCR 

Literary 

Integrate/Interpret 

62 

57 

38 

3 

MC 

Literary 

Integrate/Interpret 

57 

50 

34 

4 

MC 

Literary 

Integrate/Interpret 

63 

55 

39 

5 

MC 

Informational 

Integrate/Interpret 

76 

68 

52 

Houston 

1 

MC 

Informational 

Integrate/Interpret 

74 

64 

61 

2 

MC 

Literary 

Integrate/Interpret 

61 

56 

47 

3 

SCR 

Literary 

Integrate/Interpret 

62 

57 

49 

4 

MC 

Informational 

Integrate/Interpret 

73 

66 

61 

5 

MC 

Informational 

Integrate/Interpret 

63 

55 

51 

Jefferson 

1 

SCR 

Informational 

Critique/Evaluate 

45 

38 

37 

2 

MC 

Literary 

Integrate/Interpret 

47 

39 

32 

3 

SCR 

Literary 

Integrate/Interpret 

32 

29 

21 

4 

MC 

Informational 

Integrate/Interpret 

61 

58 

52 
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5 

MC 

Informational 

Integrate/Interpret 

78 

70 

71 

Los Angeles 

1 

SCR 

Informational 

Integrate/Interpret 

41 

32 

18 

2 

MC 

Informational 

Integrate/Interpret 

70 

64 

46 

3 

MC 

Informational 

Integrate/Interpret 

73 

64 

53 

4 

MC 

Informational 

Locate/Recall 

70 

61 

48 

5 

MC 

Informational 

Locate/Recall 

50 

44 

30 

Miami-Dade 

County 

1 

MC 

Literary 

Integrate/Interpret 

65 

61 

58 

2 

MC 

Informational 

Integrate/Interpret 

63 

55 

57 

3 

MC 

Literary 

Integrate/Interpret 

77 

71 

71 

4 

MC 

Literary 

Locate/Recall 

59 

50 

51 

5 

MC 

Informational 

Locate/Recall 

66 

60 

60 

Milwaukee 

1 

MC 

Informational 

Integrate/Interpret 

73 

64 

41 

2 

MC 

Informational 

Integrate/Interpret 

78 

70 

49 

3 

MC 

Literary 

Locate/Recall 

59 

50 

31 

4 

MC 

Informational 

Locate/Recall 

50 

44 

24 

5 

MC 

Literary 

Locate/Recall 

58 

53 

32 

New York City 

1 

MC 

Literary 

Integrate/Interpret 

64 

52 

42 

2 

SCR 

Informational 

Integrate/Interpret 

41 

32 

31 

3 

MC 

Informational 

Locate/Recall 

60 

54 

46 

4 

MC 

Literary 

Locate/Recall 

50 

43 

40 

5 

MC 

Informational 

Locate/Recall 

66 

60 

56 

Philadelphia 

1 

MC 

Literary 

Integrate/Interpret 

64 

52 

35 

2 

MC 

Informational 

Integrate/Interpret 

74 

64 

50 

3 

MC 

Informational 

Integrate/Interpret 

58 

50 

34 

4 

MC 

Informational 

Locate/Recall 

60 

53 

32 

5 

MC 

Informational 

Locate/Recall 

61 

55 

37 

San Diego 

1 

MC 

Informational 

Integrate/Interpret 

68 

64 

54 

2 

MC 

Informational 

Integrate/Interpret 

76 

68 

63 

3 

MC 

Literary 

Integrate/Interpret 

49 

44 

39 

4 

MC 

Informational 

Locate/Recall 

50 

44 

40 

5 

MC 

Informational 

Locate/Recall 

43 

35 

34 
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Table G2. Characteristics of the Top Five Differentially Most Difficult Items in 2009 Grade 8 
NAEP Reading Assessment, by District 







Percent Correct 

District/ 

jurisdiction 

Item 

Type 

Subscale 

Objective 

National 

Public 

LC 

District 

Atlanta 

1 

MC 

Literary 

Integrate/Interpret 

67 

60 

34 

2 

MC 

Informational 

Locate/Recall 

67 

61 

44 

3 

MC 

Literary 

Integrate/Interpret 

53 

46 

29 

4 

SCR 

Informational 

Critique/Evaluate 

74 

70 

52 

5 

MC 

Informational 

Integrate/Interpret 

81 

77 

61 

Austin 

1 

SCR 

Literary 

Integrate/Interpret 

54 

48 

43 

2 

ECR 

Literary 

Integrate/Interpret 

48 

43 

38 

3 

MC 

Literary 

Integrate/Interpret 

53 

46 

43 

4 

MC 

Literary 

Integrate/Interpret 

81 

73 

72 

5 

SCR 

Literary 

Integrate/Interpret 

70 

64 

60 

Baltimore 

1 

MC 

Informational 

Integrate/Interpret 

60 

53 

27 

2 

MC 

Literary 

Integrate/Interpret 

67 

60 

39 

3 

MC 

Informational 

Integrate/Interpret 

75 

67 

52 

4 

MC 

Informational 

Integrate/Interpret 

73 

65 

52 

5 

MC 

Literary 

Integrate/Interpret 

53 

46 

30 

Boston 

1 

MC 

Informational 

Integrate/Interpret 

56 

50 

37 

2 

MC 

Literary 

Integrate/Interpret 

67 

60 

49 

3 

MC 

Literary 

Integrate/Interpret 

81 

75 

65 

4 

MC 

Informational 

Integrate/Interpret 

61 

58 

46 

5 

ECR 

Informational 

Integrate/Interpret 

74 

71 

60 

Charlotte 

1 

MC 

Literary 

Locate/Recall 

70 

64 

55 

2 

MC 

Literary 

Critique/Evaluate 

86 

81 

73 

3 

MC 

Literary 

Integrate/Interpret 

81 

75 

70 

4 

MC 

Informational 

Critique/Evaluate 

76 

72 

65 

5 

SCR 

Informational 

Critique/Evaluate 

63 

62 

52 

Chicago 

1 

MC 

Informational 

Integrate/Interpret 

56 

50 

37 

2 

MC 

Literary 

Integrate/Interpret 

53 

46 

36 

3 

MC 

Informational 

Integrate/Interpret 

73 

65 

57 






4 

MC 

Literary 

Integrate/Interpret 

76 

70 

60 

5 

SCR 

Literary 

Integrate/Interpret 

54 

48 

38 

Cleveland 

1 

MC 

Literary 

Locate/Recall 

63 

56 

40 

2 

MC 

Literary 

Integrate/Interpret 

67 

60 

47 

3 

MC 

Informational 

Integrate/Interpret 

60 

53 

40 

4 

MC 

Literary 

Locate/Recall 

69 

67 

49 

5 

MC 

Informational 

Integrate/Interpret 

69 

64 

50 

Detroit 

1 

MC 

Informational 

Integrate/Interpret 

73 

65 

44 

2 

MC 

Literary 

Integrate/Interpret 

53 

46 

22 

3 

MC 

Informational 

Integrate/Interpret 

81 

77 

54 

4 

SCR 

Literary 

Integrate/Interpret 

59 

53 

29 

5 

MC 

Literary 

Integrate/Interpret 

76 

70 

48 

District of 
Columbia 

1 

MC 

Literary 

Integrate/Interpret 

86 

80 

63 

2 

MC 

Informational 

Integrate/Interpret 

59 

55 

36 

3 

SCR 

Literary 

Critique/Evaluate 

63 

56 

41 

4 

MC 

Informational 

Integrate/Interpret 

73 

65 

52 

5 

ECR 

Literary 

Integrate/Interpret 

61 

57 

40 






Percent Correct 

District/ 

jurisdiction 

Item 

Type 

Subscale 

Objective 

National 

Public 

LC 

District 

Fresno 

1 

MC 

Informational 

Locate/Recall 

67 

61 

38 

2 

MC 

Informational 

Locate/Recall 

72 

67 

45 

3 

MC 

Informational 

Locate/Recall 

72 

64 

49 

4 

ECR 

Informational 

Critique/Evaluate 

57 

52 

35 

5 

MC 

Informational 

Integrate/Interpret 

73 

65 

52 

Houston 

1 

SCR 

Literary 

Integrate/Interpret 

80 

70 

64 

2 

MC 

Informational 

Integrate/Interpret 

90 

84 

77 

3 

SCR 

Literary 

Integrate/Interpret 

54 

48 

36 

4 

MC 

Informational 

Integrate/Interpret 

79 

73 

65 

5 

MC 

Literary 

Integrate/Interpret 

67 

60 

52 

Jefferson 

1 

MC 

Literary 

Locate/Recall 

70 

64 

57 

2 

MC 

Literary 

Integrate/Interpret 

77 

74 

65 

3 

MC 

Literary 

Integrate/Interpret 

81 

75 

70 
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4 

SCR 

Literary 

Integrate/Interpret 

59 

53 

49 

5 

SCR 

Literary 

Critique/Evaluate 

65 

58 

56 

Los Angeles 

1 

MC 

Literary 

Locate/Recall 

65 

57 

44 

2 

MC 

Informational 

Integrate/Interpret 

71 

63 

51 

3 

MC 

Literary 

Integrate/Interpret 

61 

52 

42 

4 

MC 

Informational 

Locate/Recall 

67 

61 

48 

5 

MC 

Informational 

Locate/Recall 

63 

53 

45 

Miami-Dade 

County 

1 

MC 

Informational 

Integrate/Interpret 

90 

84 

78 

2 

SCR 

Literary 

Integrate/Interpret 

64 

56 

52 
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Table G3. Characteristics of the Top Five Differentially Most Difficult Items in 2009 Grade 4 NAEP Mathematics Assessment, by 
District 
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Objective 

Use informal probabilistic thinking to describe chance events 

Determine a simple probability from a context that includes a picture 

| Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Solve application problems involving numbers and operations 

Order/compare whole numbers, decimals, or fractions 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Graph or interpret points with whole number or letter coordinates on grids or in the first quadrant of the coordinate plane. 

Graph or interpret points with whole number or letter coordinates on grids or in the first quadrant of the coordinate plane. 

| Construct geometric figures with vertices at points on a coordinate grid 

Recognize or describe a relationship in which quantities change proportionally 

Determine appropriate size of unit of measurement in problem situation involving such attributes as length, time, capacity, or 
weight 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments | 

Solve problems involving perimeter of plane figures 

Order/compare whole numbers, decimals, or fractions 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Use informal probabilistic thinking to describe chance events 

Determine a simple probability from a context that includes a picture 

Determine appropriate size of unit of measurement in problem situation involving such attributes as length, time, capacity, or 
weight 

| Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Identify or describe real-world objects using simple plane figures and simple solid figures 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments | 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

| Solve problems involving perimeter of plane figures 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Identify place value and actual value of digits in whole numbers 

Read or interpret a single set of data 

Solve problems by estimating and computing within a single set of data 

Order/compare whole numbers, decimals, or fractions 
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Data 
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Numbers 

Numbers 

Numbers 

Algebra 

Algebra 

| Geometry 

Algebra 

Measurement 

Measurement 

Measurement | 

Measurement 

Numbers 

Numbers 

Numbers 

Data 

Data 

Measurement 

| Numbers 

>> 

u 

S 

o 

0) 

O 

Measurement | 

Measurement | 

| Measurement j 

Numbers 

Numbers 

Data 

Data 

Numbers 


Type 

MC 

MC 

MC 

MC 

MC 

MC 

MC 

SCR 

ECR 

MC 

MC 

SCR 

SCR 

MC 

MC 

MC 

MC 

MC 

MC 

MC 

MC 

SCR 

SCR 

MC 

MC 

MC 

MC 

MC 

MC 

MC 


Item 

- 

cm 

CO 

’'t 


- 

CM 

cn 


in 

- 

CM 

<T) 

■vf" 

IO 

- 

CM 

<T) 


m 

- 

CM 

cn 


in 

- 

CM 

m 


m 


District/ 

jurisdiction 

Atlanta 

Austin 

Baltimore 

Boston 

Charlotte 

Chicago 



American Institutes for Research * Council of the Great City Schools * Spring 2011 


167 



APPENDIX G 


Percent Correct 

District 

co 

C"- 

32 

24 

oo 

36 

30 

30 

00 

CN 

52 

20 

45 

63 

22 

in 

•^t 

55 

20 

24 

24 

62 

On 

23 

52 

48 

55 

VO 

27 

39 

47 

47 

53 

LC 

CO 

39 

VO 

in 

CN 

68 

VO 

68 

ov 

co 

50 

63 

cn 

59 

s 

42 

64 

59 

70 

a\ 

CO 

CN 

'it 

't 

CO 

63 

CO 

35 

59 

63 

68 

75 

48 

55 

VO 

64 

r- 

VO 

National 

Public 

43 

45 

69 

53 

48 

73 

69 

73 

46 

57 

73 

43 

67 

84 

48 

75 

67 

78 

46 

48 

5 

73 

43 

3 

70 

67 

73 

79 

49 

57 

69 

75 

74 


Objective 

Read or interpret a single set of data 

Recognize, describe, or extend numerical patterns 

Use place value to model and describe integers and decimals 

For a given set of data, complete a graph 

Represent numbers using models such as base 10 representations, number lines and two-dimensional models 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Use place value to model and describe integers and decimals 

Compose or decompose whole quantities by place value 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Order/compare whole numbers, decimals, or fractions 

Identify odd and even numbers. 

Read or interpret a single set of data 

Graph or interpret points with whole number or letter coordinates on grids or in the first quadrant of the coordinate plane. 

Recognize two-dimensional faces of three-dimensional shapes 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

For a given set of data, complete a graph 

Graph or interpret points with whole number or letter coordinates on grids or in the first quadrant of the coordinate plane. 

Solve problems by estimating and computing within a single set of data 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Explain or justify a mathematical concept or relationship 

Identify odd and even numbers. 

Read or interpret a single set of data 

Construct geometric figures with vertices at points on a coordinate grid 

Determine appropriate size of unit of measurement in problem situation involving such attributes as length, time, capacity, or weight 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Multiply whole numbers 

Divide whole numbers 

Use place value to model and describe integers and decimals 

For a given set of data, complete a graph 

Identify the images resulting from flips (reflections), slides (translations), or turns (rotations) 
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Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Solve application problems involving numbers and operations 


Objective 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Explain or justify a mathematical concept or relationship 

Assemble simple plane shapes to construct a given shape. 

Describe the effect of operations on size (whole numbers) 

Solve application problems involving numbers and operations 

Add and subtract whole numbers, or fractions with like denominators, or decimals through hundredths 

Identify place value and actual value of digits in whole numbers 

Represent numbers using models such as base 10 representations, number lines and two-dimensional models 

Graph or interpret points with whole number or letter coordinates on grids or in the first quadrant of the coordinate plane. 

Order/compare whole numbers, decimals, or fractions 

Read or interpret a single set of data 

Determine a simple probability from a context that includes a picture 

Identify the images resulting from flips (reflections), slides (translations), or turns (rotations) 

Determine appropriate size of unit of measurement in problem situation involving such attributes as length, time, capacity, or weight 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

For a given set of data, complete a graph 

For a given set of data, complete a graph 

Identify the images resulting from flips (reflections), slides (translations), or turns (rotations) 

Select or use appropriate measurement instruments such as ruler, meter stick, clock, thermometer, or other scaled instruments 

Order/compare whole numbers, decimals, or fractions 

Verify a conclusion using algebraic properties 

Identify factors of whole numbers 

Multiply whole numbers 

Graph or interpret points with whole number or letter coordinates on grids or in the first quadrant of the coordinate plane. 

Represent the probability of a given outcome using a picture or other graphic 
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Objective 

Describe relative positions of points and lines using the geometric ideas 

Model or describe rational numbers or numerical relationships using number lines and diagrams. 

Model or describe rational numbers or numerical relationships using number lines and diagrams. 

Solve problems involving conversions within the same measurement system. 

Interpret probabilities within a given context. 

Analyze a situation that involves probability of an independent event. 

Use place value to model and describe integers and decimals. 

Compare objects with respect to length, area, volume, angle measurement, weight, or mass. 

Solve problems involving conversions within the same measurement system. 

Perform computations with rational numbers. 

Select or use an appropriate type of unit for the attribute being measured such as length, area, angle, time, or volume. 

Model or describe rational numbers or numerical relationships using number lines and diagrams. 

Recognize or informally describe the effect of a transformation on two-dimensional geometric shapes. 

Draw or sketch from a written description polygons, circles, or semicircles. 

Construct or solve problems involving scale drawings. 

Read or interpret data, including interpolating or extrapolating from data 

Determine the probability of independent and dependent events. 

Recognize or informally describe the effect of a transformation on two-dimensional geometric shapes. 

Recognize or informally describe the effect of a transformation on two-dimensional geometric shapes. 

Interpret probabilities within a given context. 

Solve linear equations or inequalities. 

Perform basic operations, using appropriate tools, on linear algebraic expressions. 

Visually choose the line that best fits given a scatter plot and informally explain the meaning of the line. 

Identify functions as linear or nonlinear or contrast distinguishing properties of functions from tables, graphs, or equations. 

Identify or represent functional relationships in meaningful contexts 

Calculate, use, or interpret mean, median, mode, or range. 

Perform computations with rational numbers. 

Recognize, describe, or extend numerical and geometric patterns using tables, graphs, words, or symbols. 

Select or use an appropriate type of unit for the attribute being measured such as length, area, angle, time, or volume. 

Interpret relationships between symbolic linear expressions and graphs of lines by identifying and computing slope and 
intercepts 

Read or interpret data, including interpolating or extrapolating from data 

Identify lines of symmetry in plane figures or recognize and classify types of symmetries of plane figures. 
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Determine the probability of independent and dependent events. 

Draw or sketch from a written description polygons, circles, or semicircles. 

Select or use an appropriate type of unit for the attribute being measured such as length, area, angle, time, or volume. 


Objective 

Describe relative positions of points and lines using the geometric ideas 

Perform computations with rational numbers. 

Identify, define, or describe geometric shapes in the plane and in three-dimensional space given a visual representation. 

Perform basic operations, using appropriate tools, on linear algebraic expressions. 

Construct or solve problems involving scale drawings. 

Use place value to model and describe integers and decimals. 

Interpret relationships between symbolic linear expressions and graphs of lines by identifying and computing slope and 
intercepts 

Compare objects with respect to length, area, volume, angle measurement, weight, or mass. 

Determine the probability of independent and dependent events. 

Interpret probabilities within a given context. 

Read or interpret data, including interpolating or extrapolating from data 

Select or use an appropriate type of unit for the attribute being measured such as length, area, angle, time, or volume. 

Use place value to model and describe integers and decimals. 

Write algebraic expressions, equations, or inequalities to represent a situation. 

Identify, define, or describe geometric shapes in the plane and in three-dimensional space given a visual representation. 

Explain or justify a mathematical concept or relationship (e.g., explain why 17 is prime). 

Perform computations with rational numbers. 

Use place value to model and describe integers and decimals. 

Graph or interpret points represented by ordered pairs of numbers on a rectangular coordinate system. 

Solve problems involving coordinate pairs on the rectangular coordinate system. 

Recognize or informally describe the effect of a transformation on two-dimensional geometric shapes. 

Recognize or informally describe the effect of a transformation on two-dimensional geometric shapes. 

Estimate the size of an object with respect to a given measurement attribute (e.g., area). 

Use proportional reasoning to model and solve problems (including rates and scaling). 

Determine the sample space for a given situation. 
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Table HI. Changes in Grade 4 NAEP Reading Subscale and Composite Scores (Significance and 
Effect Size Measures) from 2007 to 2009, by District 14 



Composite 

Information 

Literary 

State/jurisdiction 

Effect Significance 
size 

Effect 

size 

Significance 

Effect 

size 

Significance 

National Public 

0.00 


0.01 

<— > 

-0.01 


Large City 

0.04 

<-> 

0.06 

T 

0.03 


Atlanta 

0.06 

<— > 

0.04 

<— > 

0.07 

«— > 

Austin 

0.07 

<-> 

0.11 

<— > 

0.04 


Baltimore City 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Boston 

0.15 

T 

0.13 

<— > 

0.16 

t 

Charlotte 

0.06 

<— > 

0.05 

<— > 

0.07 

«— > 

Chicago 

0.05 


0.07 

<— > 

0.02 


Cleveland 

-0.14 


-0.07 

<— > 

-0.20 


Detroit 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

District of Columbia 

0.15 

t 

0.13 

t 

0.15 

t 
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(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Houston 

0.17 

t 

0.15 

t 

0.18 

t 

Jefferson County 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Los Angeles 

0.04 

<— > 

0.03 


0.05 


Miami-Dade County 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Milwaukee 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

New York City 

0.10 

T 

0.13 

T 

0.08 


Philadelphia 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

San Diego 

0.07 

<— > 

0.09 

<— > 

0.05 


— Not available 

f Not applicable 


14 This report includes charters in the TUDA analysis according to the rules that were in place in 2007 and 2009. 
Beginning in 2009, TUDA samples included only those charter schools that each district included for the purpose of 
reporting Adequate Yearly Progress to the US Department of Education under the Elementary and Secondary 
Education Act. This rule did not exist in 2007, so the TUDA sample that year included charters without this AYP 
distinction. The inclusion or exclusion of charter schools from the TUDA samples did not affect the significance of 
the change in reported scores between 2007 and 2009, with the exception of the District of Columbia. Therefore, we 
removed charters from the District of Columbia sample in both years in order to ensure that the scores in 2007 and 
2009 for DCPS were comparable. 
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APPENDIX H 


Table H2. Changes in Grade 8 NAEP Reading Subscale and Composite Scores (Significance and 
Effect Size Measures) from 2007 to 2009, by District 



Composite 

Information 

Literary 

State/jurisdiction 

Effect 

size 

Significance 

Effect 

size 

Significance 

Effect 

size 

Significance 

National Public 

0.04 

t 

0.04 

T 

0.02 

t 

Large City 

0.07 

t 

0.04 

<— > 

0.05 

T 

Atlanta 

0.14 

t 

0.18 


0.08 


Austin 

0.11 

<— > 

0.18 

<— > 

0.04 

<— > 

Baltimore City 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Boston 

0.10 

<— >• 

0.03 

<—> 

0.14 


Charlotte 

0.00 

<— > 

0.04 

<-» 

-0.06 

<— > 

Chicago 

-0.01 

<-> 

-0.01 


-0.07 

<— >• 

Cleveland 

-0.13 

<-> 

-0.11 

<— * 

-0.23 

1 

Detroit 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

District of Columbia 

0.10 

t 

0.11 

<— » 

0.00 

<— >• 

Fresno 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Houston 

0.00 


0.04 

<— » 

-0.05 

<— >• 

Jefferson County 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Los Angeles 

0.09 

t 

0.08 

<— » 

0.03 

<— >• 

Miami-Dade County 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

Milwaukee 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

New York City 

0.09 

<— > 

0.04 


0.09 

<— >• 

Philadelphia 

(t) 

(t) 

(t) 

(t) 

(t) 

(t) 

San Diego 

0.11 


0.10 

<—> 

0.09 


— Not available 

f Not applicable 
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Table H3. Changes in Grade 4 NAEP Mathematics Subscale and Composite Scores (Significance and Effect Size Measures) from 2007 to 2009, by 
District 
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Table H4. Changes in Grade 8 NAEP Mathematics Subscale and Composite Scores (Significance and Effect Size Measures) from 2007 to 2009, by 
District 
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